[SERVER-36105] Implement server support for exhaust cursors in OP_MSG Created: 12/Jul/18  Updated: 29/Oct/23  Resolved: 21/Aug/18

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 4.1.3

Type: Task Priority: Major - P3
Reporter: Tess Avitabile (Inactive) Assignee: William Schultz (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by DRIVERS-535 Support Exhaust Cursor in OP_MSG Closed
is depended on by DRIVERS-565 Implement server support for exhaust ... Closed
Duplicate
is duplicated by SERVER-28511 OP_MSG Exhaust support for mongod ing... Closed
is duplicated by SERVER-28512 OP_MSG Exhaust support for mongos ing... Closed
Problem/Incident
Related
related to SERVER-44517 Refactor exhaust cursors on top of ex... Closed
related to SERVER-36299 Implement support for exhaust cursors... Closed
is related to SERVER-57297 Support OP_MSG exhaust cursors on mongos Closed
Backwards Compatibility: Fully Compatible
Sprint: Repl 2018-07-30, Repl 2018-08-13, Repl 2018-08-27
Participants:
Linked BF Score: 0

 Description   

The ServiceStateMachine will implement the server side of the exhaust cursor OP_MSG protocol.

  • After receiving a getMore command and running command processing, the ServiceStateMachine will check the reply message before sending it to the client.
  • If the result has an error, or a cursor ID of 0 or an empty namespace, or either of those fields does not exist or is corrupt, the reply message will be sent normally without the moreToCome bit set.
  • If the result is OK and has both valid cursor ID and namespace, the reply message will be sent with the moreToCome bit set.  The ServiceStateMachine will synthesize an OP_MSG with a getMore the same as the original getMore it was sent (including the exhaustAllowed bit), only with an ID the same as the ID of the reply.  This will result in getMore command processing being run again, and another reply being sent to the client. This reply will appear to be a reply to the previous reply.

Example: client sends a getMore with cursorId 9 for collection “mydb.mycoll” with 3 documents:

opCode: OP_MSG
requestId: 1
responseTo: 0 (not used for messages from client)
flagBits: exhaustAllowed
body: {
	getMore: 9,
	batchSize: 2
}

Server sends first response:

opCode: OP_MSG
requestId: 1001
responseTo: 1
flagBits: moreToCome
body: {
	ok: 1,
	cursor: {id: 9, ns: “mydb.mycoll”}
}
documentSequence: [doc1, doc2]

Since there are more documents, the server runs the getMore again and sends the response:

opCode: OP_MSG
requestId: 1002
responseTo: 1001
flagBits: 0
body: {
	ok: 1,
	cursor: {id: 0, ns: “mydb.mycoll”}
}
documentSequence: [doc3]

Because the cursor was exhausted, the moreToCome flag was not set and the cursor ID was 0.



 Comments   
Comment by Tess Avitabile (Inactive) [ 27/Jan/20 ]

That isn't on our roadmap currently, but we could do so if there is interest.

Comment by Matt Broadstone [ 26/Jan/20 ]

william.schultz can you provide more details on that design? Is there a plan to support exhaustAllowed on the initial command?

Comment by William Schultz (Inactive) [ 04/Oct/18 ]

jeff.yemin Correct. We currently only check for the exhaust flag on getMore commands.

Comment by Githook User [ 21/Aug/18 ]

Author:

{'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}

Message: SERVER-36105 Fix argument type in service_state_machine_test.cpp
Branch: master
https://github.com/mongodb/mongo/commit/e2f47a13f11d5a4aa2b7f23e1e0c157d90435e9e

Comment by Githook User [ 21/Aug/18 ]

Author:

{'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}

Message: SERVER-36105 Implement server support for exhaust cursors in OP_MSG
Branch: master
https://github.com/mongodb/mongo/commit/d04bafff3ff0f789e982c03b1ace55cd156f5b97

Generated at Thu Feb 08 04:42:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.