Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-1998

Queries returning large result sets fail if toArray() and count() are called on the same cursor

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.7
    • Component/s: None

      We recently updated Mongolayer (https://github.com/simpleviewinc/mongolayer/) from npm mongodb 3.1 to 3.2.7. Once our application went live our logs were filled with errors like the following:

      TypeError: Cannot read property 'write' of undefined at command (/sv_local/npm/node_modules/mongodb-core/lib/wireprotocol/command.js:94:10) at Object.getMore (/sv_local/npm/node_modules/mongodb-core/lib/wireprotocol/get_more.js:83:3) at Cursor._getmore (/sv_local/npm/node_modules/mongodb-core/lib/cursor.js:226:16) at nextFunction (/sv_local/npm/node_modules/mongodb-core/lib/cursor.js:483:10) at Cursor.next (/sv_local/npm/node_modules/mongodb-core/lib/cursor.js:766:3) at Cursor._next (/sv_local/npm/node_modules/mongodb/lib/cursor.js:216:36) at fetchDocs (/sv_local/npm/node_modules/mongodb/lib/operations/cursor_ops.js:217:12) at cursor._next (/sv_local/npm/node_modules/mongodb/lib/operations/cursor_ops.js:243:7) at handleCallback (/sv_local/npm/node_modules/mongodb-core/lib/cursor.js:204:5) at nextFunction (/sv_local/npm/node_modules/mongodb-core/lib/cursor.js:578:5)

      Tracing the problem down, the pool in command.js always didn't exist when it was called to fulfill the _getMore request. In this case we had a query that was a find() which returned 1000 docs. If we added a limit < 102 then the query would return without error, when the limit was either undefined or >=102 then we would get the error above, everytime. It was consistently reproducable.

      Now the resolution we did was updating Mongolayer with the following commit: https://github.com/simpleviewinc/mongolayer/commit/0c6f49cc8037cde6ecd01631cccc3fe9f1d7b07a . What was occurring was that we would do a cursor.toArray() and a cursor.count() in a Promise.all() so they would execute in parallel, on the same cursor. What does not make any sense to me is that the toArray() would be the one that error'd, but ONLY if there was also the count() called on the same cursor. This behavior only occurred when we were also pulling down a large set of documents.

      From looking at the docs I can't find any mention that it is not supported to call a toArray() and a count() on the same cursor. If that is forbidden then we should likely have one of the calls actually raise an error on the second call. If it's not forbidden then it's a bug in the driver somewhere. In addition, this error ONLY occurs when connecting to a replset. In our case we have a primary/secondary/arbiter replset. Consistently the ReplSet object in the mongodb-core would lack a "pool" object on it's "s" object, resulting in it erroring on the line seen in the stack trace above.

            Assignee:
            daniel.aprahamian@mongodb.com Daniel Aprahamian (Inactive)
            Reporter:
            owenallenaz Owen Allen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: