[CDRIVER-982] "limit" not used in OP_QUERY message header Created: 02/Nov/15  Updated: 13/Dec/15  Resolved: 03/Nov/15

Status: Closed
Project: C Driver
Component/s: libmongoc
Affects Version/s: None
Fix Version/s: 1.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: A. Jesse Jiryu Davis
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to CDRIVER-411 find() with a limit ignores documents... Closed
related to PHPLIB-152 Do not specify negative limit in Find... Closed
related to CDRIVER-267 Streamline limit / batchSize Closed

 Description   

The "limit" argument to mongoc_collection_find should be used for two purposes:

1. Mark a mongoc_cursor_t as "done" once it has fetched "limit" documents
2. Send limit to the server as the nToReturn field in the OP_QUERY and OP_GETMORE message headers

The driver respects #1, but not #2. Instead, it always sends nToReturn of 0 to the server, meaning, "accept server default" batch size of 101 documents at first, then 4MB of documents after.

This bug incurs a performance penalty: the final batch is always full-sized (about 4MB) even if the cursor only needs part of it.

If mongoc_collection_find's batch_size is non-0 the logic is correct; it's when batch_size is 0 (the default) that limit is incorrectly ignored when formatting OP_QUERY and OP_GETMORE:

static int32_t
_mongoc_n_return (mongoc_cursor_t * cursor)
{
   /* by default, use the batch size */
   int32_t r = cursor->batch_size;
 
   if (cursor->is_command) {
      /* commands always have n_return of 1 */
      r = 1;
   } else if (cursor->limit) {
      /* calculate remaining */
      uint32_t remaining = cursor->limit - cursor->count;
 
      /* use min of batch or remaining */
      r = BSON_MIN(r, (int32_t)remaining);
   }
 
   return r;
}



 Comments   
Comment by Githook User [ 03/Nov/15 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-982 set OP_QUERY nToReturn from limit

The "limit" argument to mongoc_collection_find should be used for two purposes:

1. Mark a mongoc_cursor_t as "done" once it has fetched "limit" documents

2. Send limit to the server as the nToReturn field in the OP_QUERY and
OP_GETMORE message headers

The driver respects #1, but not #2. Instead, it always sends nToReturn of 0 to
the server, meaning, "accept server default" batch size of 101 documents at
first, then 4MB of documents after. This bug incurs a performance penalty: the
final batch is always full-sized (about 4MB) even if the cursor only needs part
of it. If mongoc_collection_find's batch_size is non-0 the logic is correct;
it's when batch_size is 0 (the default) that limit is incorrectly ignored when
formatting OP_QUERY and OP_GETMORE.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/c123034d120880d530ae5acc800eba1aeb8fe1bb

Generated at Wed Feb 07 21:11:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.