<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 21:59:54 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[CXX-663] DBClientBase::query() returns fewer documents than specified by nToReturn</title>
                <link>https://jira.mongodb.org/browse/CXX-663</link>
                <project id="11980" key="CXX">C++ Driver</project>
                    <description>&lt;p&gt;mongo::DBClientBase::query() has the following documentation in the header file where it is declared:&lt;/p&gt;

&lt;p&gt;@param nToReturn n to return (i.e., limit).  0 = unlimited&lt;/p&gt;

&lt;p&gt;There is no other documentation explaining the true semantics of this parameter; this is the case with much of the C++ driver&apos;s (non) documentation and one normally has to look at the much better documentation of the JSON interface and try to map the JSON concepts to the C++ concepts. The JSON analogue of nToReturn parameter seems to be cursor.limit(), documented at &lt;a href=&quot;http://docs.mongodb.org/manual/reference/method/cursor.limit/#cursor.limit&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/method/cursor.limit/#cursor.limit&lt;/a&gt; as &quot;analogous to the LIMIT statement in a SQL database&quot;.&lt;/p&gt;

&lt;p&gt;Unfortunately, nToReturn is not &quot;analogous to the LIMIT statement in a SQL database&quot;. The LIMIT statement, also known as TOP, returns ALL the records matching the query criteria, if and only if the number of such records is fewer than the specified limit; otherwise, it returns exactly the limiting number of the queries. So if one gets fewer records than the specified limit, one can be sure there are NO MORE matching records.&lt;/p&gt;

&lt;p&gt;The behaviour of on nToReturn is very different. It can be described as &quot;SOME number of records will be returned, but never more than the specified number&quot;. So if one gets fewer records than the specified limit, one CANNOT be sure there are NO MORE matching records.&lt;/p&gt;

&lt;p&gt;Why does that happen? Because when nToReturn is used, the server will return only ONE batch of data. The batch will contain no more than the specified number of records, AND ALSO NO MORE THAN MaxBytesToReturnToClientAtOnce bytes per batch, which is 4 MiB (as of v3.0). This second condition is not documented anywhere; the only place which tries to say something about these limitations I have been able to find is at &lt;a href=&quot;https://docs.mongodb.org/manual/core/cursors/#cursor-batches&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.org/manual/core/cursors/#cursor-batches&lt;/a&gt; and it says this&lt;/p&gt;

&lt;p&gt;&quot;The MongoDB server returns the query results in batches. Batch size will not exceed the maximum BSON document size. &quot;&lt;/p&gt;

&lt;p&gt;This is a lie, because the max BSON doc size is 16 MiB, while the server-side limitation is 4 MiB.&lt;/p&gt;

&lt;p&gt;Why is that important? Partly because it cost me half a day to figure out why queries from a collection having a few million matching documents, with nToReturn set to 10 thousand, would only return 3-4 thousands. Yes, the document size was 1-2 KiB, running into the undocumented limitation. I can easily imagine that other people might run into this issue.&lt;/p&gt;

&lt;p&gt;More important is that the current behaviour of nToReturn makes it pretty much useless. How did I fix the problem eventually? By NOT using nToReturn and implementing the constraint on top of the driver, which is quite a bit less efficient.&lt;/p&gt;

&lt;p&gt;I am not sure what this problem really is: bad documentation, bad driver code, or bad server code, or bad design overall. But the whole point is that people really DO expect in such APIs that if the number of returned records is fewer than the limit, then there is no more matching records. Having it done differently is a very very bad surprise, and having it differently in an UNKNOWN way is much worse.&lt;/p&gt;</description>
                <environment></environment>
        <key id="228743">CXX-663</key>
            <summary>DBClientBase::query() returns fewer documents than specified by nToReturn</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="adam.midvidy">Adam Midvidy</assignee>
                                    <reporter username="viaus">Viacheslav Usov</reporter>
                        <labels>
                            <label>legacy-cxx</label>
                    </labels>
                <created>Tue, 8 Sep 2015 14:19:18 +0000</created>
                <updated>Wed, 7 Oct 2015 19:08:48 +0000</updated>
                            <resolved>Fri, 2 Oct 2015 15:06:00 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="1054279" author="adam.midvidy" created="Wed, 7 Oct 2015 19:08:48 +0000"  >&lt;p&gt;Hello Viacheslav, it looks like this driver behavior is actually erroneous. The problem is that the driver was sending an incorrect nToReturn value over the wire, which inhibited the server from creating further batches to return to the client. I am sorry that we initially misdiagnosed this issue - you can track its fix at &lt;a href=&quot;https://jira.mongodb.org/browse/CXX-699&quot; title=&quot;Queries may return incomplete result sets due to incorrect calculation of nToReturn&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CXX-699&quot;&gt;&lt;del&gt;CXX-699&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1049882" author="adam.midvidy" created="Fri, 2 Oct 2015 15:05:33 +0000"  >&lt;p&gt;Hi Viacheslav, I have closed this ticket as we will not be able to improve nToReturn semantics until MongoDB 3.2. Please comment or file another ticket if you have any further questions.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hre8c7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="673">Platform 9 (09/18/15)</customfieldvalue>
    <customfieldvalue id="688">Platform A (10/09/15)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>