Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30799

Tailable cursor with batch size periodically returns unexpected empty batches

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.5.13
    • Affects Version/s: None
    • Component/s: Querying
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Hide

      Save this to a file called repro.js at the root of the mongo repository directory:

      repro.js
      (function() {
    "use strict";
    const st = new ShardingTest({shards: 1});
    const db = st.s.getDB("test");

    assert.commandWorked(db.runCommand({create: "capped", capped: true, size: 1024}));
    assert.writeOK(db.capped.insert([{_id: 1}, {_id: 2}, {_id: 3}, {_id: 4}, {_id: 5}, {_id: 6}]));

    const findRes =
        assert.commandWorked(db.runCommand({find: "capped", tailable: true, batchSize: 2}));
    const cursorId = findRes.cursor.id;
    assert.neq(cursorId, 0);
    let getMoreRes = assert.commandWorked(
        db.runCommand({getMore: cursorId, collection: "capped", batchSize: 2}));
    // This assertion fails, zero results in this batch.
    assert.eq(getMoreRes.cursor.nextBatch.length, 2);
    st.stop();
}());

      Then run:

      python buildscripts/resmoke.py repro.js
      Show
      Save this to a file called repro.js at the root of the mongo repository directory: repro.js ( function () { "use strict" ; const st = new ShardingTest({shards: 1}); const db = st.s.getDB( "test" ); assert.commandWorked(db.runCommand({create: "capped" , capped: true , size: 1024})); assert.writeOK(db.capped.insert([{_id: 1}, {_id: 2}, {_id: 3}, {_id: 4}, {_id: 5}, {_id: 6}])); const findRes = assert.commandWorked(db.runCommand({find: "capped" , tailable: true , batchSize: 2})); const cursorId = findRes.cursor.id; assert.neq(cursorId, 0); let getMoreRes = assert.commandWorked( db.runCommand({getMore: cursorId, collection: "capped" , batchSize: 2})); // This assertion fails, zero results in this batch. assert.eq(getMoreRes.cursor.nextBatch.length, 2); st.stop(); }()); Then run: python buildscripts/resmoke.py repro.js
    • Repl 2017-09-11

      When a tailable cursor is used in combination with a batch size B, the semantics should be:

      • The initial find should return up to B results, without waiting for at least B. That is, if B = 4, but there were only 3 matching documents, it should return just those 3.
      • A getMore against a tailable cursor should return an empty batch if there are no further results.
      • A getMore against a tailable cursor with N < B new results should return N without waiting for a full B to appear to fill the batch.
      • A getMore against a tailable cursor with N >= B new results should return B, and return the rest on the next getMore (subject to the same batch size constraints).

      If using a tailable cursor against a mongod process, these are the semantics. However, if running against a mongos (which you can do with an unsharded capped collection), the last case breaks down a bit. The mongos correctly returns batches up to size B, but messes up the next getMore, incorrectly 'remembering' that the next result on that cursor should be EOF (see reproduction steps for more details).

      Specifically, the boolean '_eofNext' tracked within the AsyncResultsMerger is not reset on each getMore, so a full batch on one getMore will cause an empty batch on the next, even if there are more results to return.

            Assignee:
            charlie.swanson@mongodb.com Charlie Swanson
            Reporter:
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: