Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-83

Support the Exhaust flag on query

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Component/s: None
    • None

      There are two ways to resolve this ticket for a given driver:

      1. Raise an exception / set an error state in your driver if the user tries to set the exhaust flag or just make the flag impossible to set. This is probably what you want to do.
      2. Implement support for the flag in your driver. Keep reading for details...

      There are several drivers that specify the Exhaust flag for queries but don't actually support it properly if set. There are several things you have to ensure to support exhaust:

      1. Cursor still needs to handle the equivalent of the initial query response and one or more subsequent responses. However, actual getMore messages are not sent to the server. You just receive off the same socket again.
      2. The responses to these implicit "getMores" all must be read from the same socket, not just the same server.
      3. If the user doesn't iterate through all the results, the driver must either close the socket or drain the socket of the remaining getMore response. Otherwise, the next time the socket is read from, you'll be reading the wrong thing. You can tell when you're done because the last message will have cursorId = 0.
      4. mongos does not support the exhaust flag. Furthermore, it does not raise an error if the exhaust flag is set. Your driver will just hang trying to read the second response. You must raise an error if the user tries to use exhaust when connected to mongos.
      5. When iterating the exhaust cursor, the response id returned for each batch is equivalent to the request id the server returned with the previous batch.
      6. Exhaust and limit are incompatible. Limit is a client side concept that requires altering "ntoreturn" in each getMore call and sending a kill cursors command to the server when you have received "limit" number of documents. Since exhaust cursors don't use getMore calls and don't support kill cursors, there is no sane way to support limit. Your driver should raise an error if exhaust and limit are specified together.

      Note also that like any query, the total number of documents returned can be arbitrarily large, so care should be taken to only require memory proportional to each chunk.

      If the driver does not support the exhaust flag properly, but does support setting the query flags, it should at minimum throw an exception if the Exhaust flag is set by the user, since otherwise the errors you get are hard to diagnose.

            Assignee:
            barrie Barrie Segal
            Reporter:
            jeff.yemin@mongodb.com Jeffrey Yemin
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: