<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:51:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[JAVA-226] serialization of DBList is slow since it involves O(n^2) operation when creating keySet.</title>
                <link>https://jira.mongodb.org/browse/JAVA-226</link>
                <project id="10006" key="JAVA">Java Driver</project>
                    <description>&lt;p&gt;reported by FourQuare&lt;/p&gt;

&lt;p&gt;---------- Forwarded message ----------&lt;br/&gt;
From: Jon Hoffman &amp;lt;jon@foursquare.com&amp;gt;&lt;br/&gt;
Date: Sat, Dec 4, 2010 at 12:33 PM&lt;br/&gt;
Subject: Re: slow query construction (java driver issue)&lt;br/&gt;
To: harryh &amp;lt;harryh@foursquare.com&amp;gt;&lt;br/&gt;
Cc: Eliot Horowitz &amp;lt;eliot@10gen.com&amp;gt;, &quot;Brendan W. McAdams&quot; &amp;lt;brendan@10gen.com&amp;gt;&lt;/p&gt;


&lt;p&gt;should be O(1)&lt;/p&gt;


&lt;p&gt;On Dec 4, 2010, at 1:18 PM, harryh wrote:&lt;/p&gt;

&lt;p&gt;&amp;gt; Ya, I &lt;b&gt;just&lt;/b&gt; found that as well.  It should be O&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/thumbs_down.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;.&lt;br/&gt;
&amp;gt;&lt;br/&gt;
&amp;gt; On Sat, Dec 4, 2010 at 1:17 PM, Jon Hoffman &amp;lt;jon@foursquare.com&amp;gt; wrote:&lt;br/&gt;
&amp;gt;&amp;gt; BasicBSONList.keySet is O(n^2)&lt;br/&gt;
&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt; On Dec 4, 2010, at 1:04 PM, harryh wrote:&lt;br/&gt;
&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; What&apos;s Brendan&apos;s @10gen e-mail?  This is totally his territory.&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; We found a bug in foursquare last night where under certain rare&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; circumstances we were constructing a mongo query with a 50 thousand&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; item $in clause.  This is obviously bad on our part and we fixed it,&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; but I&apos;m also pretty sure it uncovered an efficiency problem in the&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; mongo java driver.  It was taking on the order of 10 seconds to&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; &lt;b&gt;construct&lt;/b&gt; the query to send to the database.  Relavent stack trace&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; is below.  I&apos;ve only just begun looking through the code, but I&apos;m&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; already struck by the N^2 code that executes as the&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; c.m.util.OrderedSet is filled up with the 50k ids in question.  That&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; might be worth changing?&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; -harryh&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; &quot;710173487@qtp-768288241-81&quot; prio=10 tid=0x00007fb2c002c000 nid=0x778e&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt; runnable &lt;span class=&quot;error&quot;&gt;&amp;#91;0x00007fb2ae0e2000&amp;#93;&lt;/span&gt;&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;   java.lang.Thread.State: RUNNABLE&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at java.util.ArrayList.indexOf(ArrayList.java:216)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at java.util.ArrayList.contains(ArrayList.java:199)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.util.OrderedSet.add(OrderedSet.java:35)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.types.BasicBSONList.keySet(BasicBSONList.java:137)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder.putObject(BSONEncoder.java:109)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder._putObjectField(BSONEncoder.java:154)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder.putObject(BSONEncoder.java:119)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder._putObjectField(BSONEncoder.java:154)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder.putObject(BSONEncoder.java:119)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at org.bson.BSONEncoder.putObject(BSONEncoder.java:65)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.OutMessage._appendQuery(OutMessage.java:70)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.OutMessage.query(OutMessage.java:55)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:221)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBCursor._check(DBCursor.java:252)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBCursor._hasNext(DBCursor.java:372)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBCursor._fill(DBCursor.java:440)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBCursor.toArray(DBCursor.java:470)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.mongodb.DBCursor.toArray(DBCursor.java:459)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$$anonfun$findAll$4.apply(MongoMetaRecord.scala:154)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$$anonfun$findAll$4.apply(MongoMetaRecord.scala:146)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.MongoDB$.useCollection(Mongo.scala:155)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$class.useColl(MongoMetaRecord.scala:54)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.useColl(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$class.findAll(MongoMetaRecord.scala:146)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.findAll(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$class.findAll(MongoMetaRecord.scala:133)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.com$foursquare$record$FSMongoMetaRecord$$super$findAll(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.record.FSMongoMetaRecord$$anonfun$findAll$4.apply(FSMongoRecord.scala:167)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.record.FSMongoMetaRecord$$anonfun$findAll$4.apply(FSMongoRecord.scala:167)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.record.FSMongoMetaRecord$class.logTime(FSMongoRecord.scala:198)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.logTime(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.record.FSMongoMetaRecord$class.findAll(FSMongoRecord.scala:167)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.findAll(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at net.liftweb.mongodb.record.MongoMetaRecord$class.findAll(MongoMetaRecord.scala:182)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.Badge$.findAll(Badge.scala:69)&lt;br/&gt;
&amp;gt;&amp;gt;&amp;gt;       at com.foursquare.model.User$$anonfun$13.apply(User.scala:665)&lt;br/&gt;
&amp;gt;&amp;gt;&lt;br/&gt;
&amp;gt;&amp;gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="13865">JAVA-226</key>
            <summary>serialization of DBList is slow since it involves O(n^2) operation when creating keySet.</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="antoine">Antoine Girbal</assignee>
                                    <reporter username="antoine">Antoine Girbal</reporter>
                        <labels>
                    </labels>
                <created>Sun, 5 Dec 2010 08:11:00 +0000</created>
                <updated>Wed, 22 Dec 2010 23:58:14 +0000</updated>
                            <resolved>Mon, 6 Dec 2010 22:17:16 +0000</resolved>
                                                    <fixVersion>2.4</fixVersion>
                                                        <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="21123" author="auto" created="Mon, 6 Dec 2010 22:32:02 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;login&apos;: &apos;agirbal&apos;, &apos;name&apos;: &apos;agirbal&apos;, &apos;email&apos;: &apos;antoine@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/JAVA-226&quot; title=&quot;serialization of DBList is slow since it involves O(n^2) operation when creating keySet.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;JAVA-226&quot;&gt;&lt;del&gt;JAVA-226&lt;/del&gt;&lt;/a&gt;: - improved StringRangeSet to use static Strings for 1st 100 numbers&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;deleted OrderedSet class, and made other classes use LinkedHashSet instead&lt;br/&gt;
/mongodb/mongo-java-driver/commit/9646e202220cae81ddc3cb270e26ed0979004f9f&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="21121" author="antoine" created="Mon, 6 Dec 2010 22:17:16 +0000"  >&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;imported patch from fourquare, to replace OrderedSet with StringRangeSet&lt;/li&gt;
	&lt;li&gt;improved StringRangeSet to use static Strings for 1st 100 numbers&lt;/li&gt;
	&lt;li&gt;deleted OrderedSet class, and made other classes use LinkedHashSet instead&lt;/li&gt;
&lt;/ul&gt;
</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrhcjz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14834</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>