<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:56:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-1285] Mongo cursor return less entities than it has.</title>
                <link>https://jira.mongodb.org/browse/SERVER-1285</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When I try fetch many big entities from cursor, it&apos;s return less than it has. &lt;br/&gt;
When I fetch not so match big entities, or small entities, cursor work correctly. &lt;br/&gt;
The more memory the object occupies, the less its returns cursor.&lt;br/&gt;
Maybe cursor have memory limit. And if entities summary size more than it limit, cursor return not all entities.&lt;/p&gt;

&lt;p&gt;Mongo version 1.4.3&lt;/p&gt;

&lt;p&gt;Examples:&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find().skip(0).limit(400).toArray();&lt;br/&gt;
&amp;gt; cursor.length;&lt;br/&gt;
289&lt;/p&gt;

&lt;p&gt;But I have more than 289 entities:&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find().skip(0).limit(400);&lt;br/&gt;
&amp;gt; cursor.count(true);&lt;br/&gt;
400&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find().skip(400).limit(100).toArray();&lt;br/&gt;
&amp;gt; cursor.length;&lt;br/&gt;
100&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find().skip(400).limit(400).toArray();&lt;br/&gt;
&amp;gt; cursor.length;&lt;br/&gt;
283&lt;/p&gt;

&lt;p&gt;If I run query witout limit, cursor return less entities:&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find().toArray();&lt;br/&gt;
&amp;gt; cursor.length;&lt;br/&gt;
71&lt;br/&gt;
&amp;gt; var cursor = db.test_coll.find();&lt;br/&gt;
&amp;gt; cursor.count(true);&lt;br/&gt;
835&lt;/p&gt;

&lt;p&gt;It problem I have in mongo console and in PHP API: &lt;br/&gt;
$data = $mongo_collection_handler-&amp;gt;find(array())&lt;del&gt;&amp;gt;skip(0)&lt;/del&gt;&amp;gt;limit(400);&lt;br/&gt;
echo $data-&amp;gt;count(true);  // return 835&lt;br/&gt;
$counter = 0;&lt;br/&gt;
foreach ($data as $value){&lt;br/&gt;
 $counter++;&lt;br/&gt;
}&lt;br/&gt;
echo $counter; // return 289&lt;/p&gt;</description>
                <environment></environment>
        <key id="12232">SERVER-1285</key>
            <summary>Mongo cursor return less entities than it has.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="eliot">Eliot Horowitz</assignee>
                                    <reporter username="pavel.pipkin">Pavel Pipkin</reporter>
                        <labels>
                    </labels>
                <created>Wed, 23 Jun 2010 09:24:56 +0000</created>
                <updated>Fri, 30 Mar 2012 14:37:10 +0000</updated>
                            <resolved>Sat, 17 Sep 2011 01:58:00 +0000</resolved>
                                    <version>1.4.3</version>
                                                    <component>Querying</component>
                                        <votes>4</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="100139" author="bwaldvogel" created="Mon, 19 Mar 2012 15:30:10 +0000"  >&lt;p&gt;After a longer debugging session I found the bug in the code. Surprisingly, the bug was already known and fixed recently: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4680&quot; title=&quot;Inconsistent query results on large data and result sets&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4680&quot;&gt;&lt;del&gt;SERVER-4680&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="55106" author="bwaldvogel" created="Mon, 19 Sep 2011 16:37:25 +0000"  >&lt;p&gt;I can still reproduce the problem with mongodb 2.0.0.&lt;/p&gt;

&lt;p&gt;I&apos;ve written a small Java program that lets me (and hopefully you) easily reproduce the problem:&lt;br/&gt;
&lt;a href=&quot;https://gist.github.com/1226902#file_multiple_id_query_test.java&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://gist.github.com/1226902#file_multiple_id_query_test.java&lt;/a&gt;&lt;br/&gt;
The output looks for me like:&lt;br/&gt;
&lt;a href=&quot;https://gist.github.com/1226902#file_output.log&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://gist.github.com/1226902#file_output.log&lt;/a&gt;&lt;br/&gt;
And the collection stats:&lt;br/&gt;
&lt;a href=&quot;https://gist.github.com/1226902#file_collection_stats.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://gist.github.com/1226902#file_collection_stats.js&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="54901" author="eliot" created="Sat, 17 Sep 2011 01:58:00 +0000"  >&lt;p&gt;After the repair - please let me know if you see it again.&lt;/p&gt;</comment>
                            <comment id="52776" author="spajan" created="Wed, 7 Sep 2011 14:09:50 +0000"  >&lt;p&gt;It&apos;s being prepared for production use. I&apos;ll see if we can do a repair on it.&lt;/p&gt;</comment>
                            <comment id="52686" author="eliot" created="Tue, 6 Sep 2011 21:48:17 +0000"  >&lt;p&gt;Is this a production db?&lt;br/&gt;
If you could run a repair on it would be very helpful to know its a data issue or something else&lt;/p&gt;</comment>
                            <comment id="52652" author="spajan" created="Tue, 6 Sep 2011 19:34:52 +0000"  >&lt;p&gt;We are using journaling.&lt;/p&gt;</comment>
                            <comment id="52651" author="spajan" created="Tue, 6 Sep 2011 19:33:30 +0000"  >&lt;p&gt;Thanks for the quick response.&lt;/p&gt;

&lt;p&gt;Yes, running the same queries multiple times lead to the same results. Note that I get MORE results when asking for 9900 entries than when asking for 10000 entries. &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/warning.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;


&lt;p&gt;&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).sort({_id:1}).limit(10000).itcount()&lt;br/&gt;
9818&lt;br/&gt;
&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).sort({_id:1}).limit(10000).itcount()&lt;br/&gt;
9818&lt;br/&gt;
&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).sort({_id:1}).limit(9900).itcount() &lt;br/&gt;
9900&lt;br/&gt;
&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).sort({_id:1}).limit(9900).itcount()&lt;br/&gt;
9900&lt;/p&gt;

&lt;p&gt;I think we are using journaling, will check. We haven&apos;t tried upgrading to 1.8.3 yet, but will do that.&lt;/p&gt;</comment>
                            <comment id="52563" author="eliot" created="Tue, 6 Sep 2011 14:56:51 +0000"  >&lt;p&gt;Its impossible to say without more information.&lt;br/&gt;
Can you run the same query multiple times?&lt;br/&gt;
Can you try 1.8.3?&lt;br/&gt;
Also, are you running with journalling, and if not was there an unclean shutdown?&lt;/p&gt;</comment>
                            <comment id="52557" author="spajan" created="Tue, 6 Sep 2011 14:38:15 +0000"  >&lt;p&gt;I have the same issue. The database has about 2 billion instances with higher _id than 2000120855.&lt;/p&gt;

&lt;p&gt;&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).limit(10000).itcount()                      &lt;br/&gt;
6209&lt;br/&gt;
&amp;gt; db.instance.find({_id: {$gt: 2000120855}}).sort({_id:1}).limit(10000).itcount()&lt;br/&gt;
9818&lt;br/&gt;
&amp;gt; db.version()&lt;br/&gt;
1.8.1&lt;/p&gt;

&lt;p&gt;This results are consistent, i.e. the queries return the same results if run repeatedly.&lt;/p&gt;

&lt;p&gt;Is this fixed between 1.8.1 and 1.8.3?&lt;/p&gt;</comment>
                            <comment id="52126" author="eliot" created="Fri, 2 Sep 2011 04:51:53 +0000"  >&lt;p&gt;There was an issue with 1.4.3 that might be related.&lt;br/&gt;
Can you try 1.8.3?&lt;/p&gt;</comment>
                            <comment id="26120" author="bwaldvogel" created="Thu, 17 Mar 2011 17:44:28 +0000"  >&lt;p&gt;I&apos;ve copy/pasted the log of mongos -v here: &lt;a href=&quot;https://gist.github.com/874740&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://gist.github.com/874740&lt;/a&gt;&lt;br/&gt;
I see nothing special.&lt;/p&gt;

&lt;p&gt;Actually I&apos;ve used the Java driver to count the number of items since the shell becomes very slow with such big queries.&lt;/p&gt;

&lt;p&gt;collection.find(query-with-4000-ids).limit(4000).size();&lt;br/&gt;
&amp;gt; returns 4000&lt;/p&gt;

&lt;p&gt;int count=0;&lt;br/&gt;
for (DBObject o : collection.find(query-with-4000-ids)) {&lt;br/&gt;
    count++;&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; count is 4000.&lt;/p&gt;

&lt;p&gt;int count=0;&lt;br/&gt;
for (DBObject o : collection.find(query-with-4000-ids).limit(4000)) {&lt;br/&gt;
    count++;&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; count is 3956.&lt;/p&gt;

&lt;p&gt;The query looks like:&lt;/p&gt;

&lt;p&gt;{ &quot;_id&quot; : &lt;/p&gt;
{ &quot;$in&quot; : [ 554795360, 554795363, ... ] }
&lt;p&gt; }&lt;/p&gt;

&lt;p&gt;I double checked that the list of IDs doesn&apos;t contain duplicates.&lt;/p&gt;</comment>
                            <comment id="26108" author="eliot" created="Thu, 17 Mar 2011 16:44:27 +0000"  >&lt;p&gt;Can you increase verbosity on the mongos and try again?&lt;br/&gt;
And send the full query?&lt;/p&gt;

&lt;p&gt;Also try:&lt;/p&gt;

&lt;p&gt;db.coll.find( query-with-10000-ids ).limit(10000).itcount()&lt;/p&gt;</comment>
                            <comment id="26096" author="bwaldvogel" created="Thu, 17 Mar 2011 16:12:52 +0000"  >&lt;p&gt;still fails on 1.8.0. the logfiles contain nothing.&lt;/p&gt;</comment>
                            <comment id="26082" author="bwaldvogel" created="Thu, 17 Mar 2011 13:48:11 +0000"  >&lt;p&gt;I&apos;m able to reproduce this problem on mongo 1.6.5 and mongo-java-driver 2.4/2.5:&lt;/p&gt;

&lt;p&gt;My setup has two shards.&lt;/p&gt;

&lt;p&gt;&amp;gt; db.coll.find( query-with-10000-ids ).length() returns 10000&lt;br/&gt;
&amp;gt; db.coll.find( query-with-10000-ids ).limit(10000) returns ~5000&lt;br/&gt;
&amp;gt; db.coll.find( query-with-5000-ids ).limit(5000) returns ~2500&lt;/p&gt;

&lt;p&gt;There&apos;s also a thread in the mongo user group that discussed the same issue: &lt;a href=&quot;http://groups.google.com/group/mongodb-user/browse_thread/thread/c80f62b62650eb1a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user/browse_thread/thread/c80f62b62650eb1a&lt;/a&gt;&lt;br/&gt;
I&apos;ll try it on 1.8.0 in a few minutes after the upgrade is done.&lt;/p&gt;</comment>
                            <comment id="24135" author="eliot" created="Thu, 17 Feb 2011 19:50:16 +0000"  >&lt;p&gt;Can you try with 1.7.6?  A number of fixed.&lt;/p&gt;</comment>
                            <comment id="18545" author="eliot" created="Sun, 26 Sep 2010 04:43:14 +0000"  >&lt;p&gt;Can you send db.printShardingStatus()&lt;/p&gt;</comment>
                            <comment id="18360" author="onewhomikes" created="Tue, 21 Sep 2010 01:17:05 +0000"  >&lt;p&gt;I&apos;m seeing similar behavior in version:&lt;br/&gt;
db version v1.7.1-pre-, pdfile version 4.5&lt;br/&gt;
Mon Sep 20 18:14:39 git version: 8d53011001891a36d5f7abf6b5c2117bda5be889&lt;/p&gt;

&lt;p&gt;The following is from a mongo console connected to a mongos process and sharded database.  Also, there is an index on the &quot;date&quot; field.&lt;/p&gt;

&lt;p&gt; &amp;gt; var query = { date: &lt;/p&gt;
{ &apos;$gte&apos;: 1284793200000, &apos;$lte&apos;: 1284793300000 }
&lt;p&gt; } &lt;br/&gt;
 &amp;gt; db.log.count( query ) &lt;br/&gt;
2501 &lt;br/&gt;
 &amp;gt; db.log.find( query ).length() &lt;br/&gt;
2473 &lt;br/&gt;
 &amp;gt; db.log.find( query ).limit(5000).length() &lt;br/&gt;
2344 &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>17.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 21 Sep 2010 01:17:05 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 48 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 48 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10024"><![CDATA[FreeBSD]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bwaldvogel</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>spajan</customfieldvalue>
            <customfieldvalue>onewhomikes</customfieldvalue>
            <customfieldvalue>pavel.pipkin</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrpkx3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrikm7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>21992</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0cfb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>