[CDRIVER-2765] new driver session support breaks application that is not designed with session Created: 25/Jul/18  Updated: 28/Oct/23  Resolved: 16/Aug/18

Status: Closed
Project: C Driver
Component/s: libmongoc
Affects Version/s: 1.11.0
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Ben Cheong Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Application using older c driver, e.g. 1.6, which does not support session is allowed to have following flow:

  1. using older version c driver api, get a client through client pool
  2. construct a find bson command
  3. issue the command using collectionCommandWithOpts
  4. bson containing cursor id information is returned
  5. construct a getMore bson command using the cursor id
  6. issue the command using collectionCommandWithOpts
  7. all batches are successfully returned

However, using c driver with newer version, the above flow is broken. Server will sometimes returns error in step (6) - "Cannot run getMore on cursor ..., which was created in session ..."

The followings are some points about the causes:

  • implicit session is implemented for cursor, however, its only implemented for mongoc_cursor_t returned api
  • session pool implemented in c driver will have chance to cause the getMore command in (6) to acquire a new session or reused another pooled session, which is not same as the session used by find command in (3)

From the specification defined, with backward compatibility consideration, existing application is not required to make changes to run properly.

Should we treat this issue as bug or should we change our application to adapt the driver session change?



 Comments   
Comment by Kevin Albertson [ 16/Aug/18 ]

Hi Ben, I'm closing this ticket, but feel free to re-open if you're still having issues.

Comment by Ben Cheong [ 27/Jul/18 ]

Hi Davis, thanks for the suggestion. We will try it.

Comment by A. Jesse Jiryu Davis [ 27/Jul/18 ]

I see, interesting. You can try calling mongoc_client_start_session in your program. If it fails that means the MongoDB server doesn't support sessions. If it succeeds, then you can explicitly add the session to each incoming user command with mongoc_client_session_append (session, opts). That way you're assured that "find" and "getMore" use the same session id.

The server reaps sessions that have been idle for 30 minutes. So, track your session's age, and before you use it check if it's been idle for 29+ minutes. If it has, destroy it and create a new one.

Comment by Ben Cheong [ 27/Jul/18 ]

We actually understand the helper functions are convenient to use, but it just increased the difficulty for our use case.

our use case is like this:

  1. input: user input documented command by typing, e.g. { find: ... }
  2. application: process the command by issuing c driver api, interacting with server
  3. output: bson result from the call, and display to user

The whole feature is to let user tests the commands listed in mongodb documentation, we will need to do a parsing of the input and forward to different c driver helper calls. We could use that approach, but just found that using mongoc_collection_command_with_opts is more convenient for this use case.

Anyway, we will review again the approach, thanks for the tips!

Comment by A. Jesse Jiryu Davis [ 26/Jul/18 ]

The best practice is to use C Driver helper functions instead of "command" functions whenever possible. So, use mongoc_collection_find_with_opts instead of running the "find" command directly with mongoc_collection_command_with_opts, and use mongoc_collection_insert_many instead of running the "insert" command directly, use mongoc_collection_aggregate instead of running the "aggregate" command directly, etc.

The helper functions are provided to you to implement special logic specific to those commands. For example, the "aggregate" command requires that you send the same session with both "aggregate" and "getMore", the same as the "find" command does. The mongoc_collection_aggregate helper implements this session logic for you, but if you run the "aggregate" command directly you'll have difficulty.

Similarly for the "insert" command: the mongoc_collection_insert_many helper ensures that you properly split multiple documents into batches of 1000 documents or 16 MB each. This is difficult to implement yourself if you call "insert" with mongoc_collection_command_with_opts.

I recommend that you use the C Driver's helper functions whenever we provide them to you. We provide them for a reason!

Comment by Ben Cheong [ 26/Jul/18 ]

Hi Davis, our application intention is to use documented commands solely, and show these commands to users according to the MongoDB documentation (e.g. https://docs.mongodb.com/manual/reference/method/db.collection.find and https://docs.mongodb.com/manual/reference/command/getMore/#dbcmd.getMore)

We used mongoc_collection_command_with_opts instead of mongoc_collection_find_with_opts, since such call allows us to use the same call for all documented commands, and allows us to have a general coding structure to handle our feature.

We are not aware of session changes until we tried to upgrade the c driver to a newer version. If c driver does not handle implicit session for find + getMore commands case, we have to either overhaul our internal to use cursor returned approach, which we want to avoid, or we have to use explicit session on top of all current flows.

Do you expect application use explicit session for all session supported c driver api whenever possible, as a good practice?

Comment by A. Jesse Jiryu Davis [ 25/Jul/18 ]

Hi, thanks for the report. Can you tell me why you're constructing find and getMore commands using mongoc_collection_command_with_opts? We intend for you to use mongoc_collection_find_with_opts to create a cursor, and mongoc_cursor_next to retrieve results.

Comment by Ben Cheong [ 25/Jul/18 ]

typo:

collectionCommandWithOpts is mongoc_collection_command_with_opts in c driver api

Generated at Wed Feb 07 21:16:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.