Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-91814

Investigate how exhaust cursors should be handled

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Networking & Observability
    • N&O Prioritized List

      Recently, the query team found a bug SERVER-91133 where the exhaust cursor was being closed by the shell before it was fully exhausted instead of being closed by the server when there no pending replies on the connection. This caused the network to still relay the closed cursor's result to the client till it was fully exhausted.

      The fix for the issue on the query side is to ensure that the shell does not close the cursor when limit is reached. 

      This led to the following question: "What should the ideal behavior be for an exhaust cursor?" max.hirschhorn@mongodb.com wrote this script where we are unsure of the correct/ intended behavior. 

      /* ... insert 1 billion documents ... */
      var cursor = db.collection.find().batchSize(1).addOption(DBQuery.Option.exhaust);
      cursor.next(); // runs find command
      cursor.next(); // runs getMore command with moreToCome=true
      /* ... server begins filling network buffer ... */
      cursor.next(); // read out of the network buffer
      cursor.next(); // read out of the network buffer
      /* important: network buffer is full and cursor hasn't sent moreToCome=false OP_MSG reply yet */
      
      cursor.close(); // is this allowed?
      db.runCommand("ping"); // attempt to reuse the connection for a different command now 

      Can we close that cursor? Should we throw away this connection on close and establish a new one? What layer this should be handled in (pymongo has related questions, but unsure of where it's being handled)? 
       
      Some additional context from Max:
       
      Maybe to ask it another way - what should an application do to the following: # Create an exhaust create

      1. Read from the exhaust cursor for 5 seconds (where the network buffer was full and the server didn't reach cursorExhausted==true)
      2. Abandon reading the exhaust cursor
      3. Run another command on the same network connection

      From pymongo:

      Exhaust Cursors may require changes to how we close Connections in the future, specifically to add a way to close and remove from its pool a Connection which has unread exhaust messages.https://github.com/mongodb/specifications/blob/663db4db13c8acf415bb97259bf7f1909b32a670/source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.md#add-support-for-op_msg-exhaustallowed

            Assignee:
            Unassigned Unassigned
            Reporter:
            adi.agrawal@mongodb.com Adi Agrawal
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: