[SERVER-32064] A logical session id should be included in all command requests from the mongo shell Created: 21/Nov/17  Updated: 30/Oct/23  Resolved: 25/Jun/18

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: 3.6.9, 4.0.1, 4.1.1

Type: Improvement Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Jack Mulrow
Resolution: Fixed Votes: 3
Labels: ShardingTechDebt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Documented
is documented by DOCS-11835 Docs for SERVER-32064: A logical sess... Closed
Duplicate
is duplicated by SERVER-34153 MongoDB shell doesn't create a sessio... Closed
Gantt Dependency
has to be done before SERVER-35447 Implement exceptions to sending sessi... Closed
Problem/Incident
Related
is related to SERVER-34204 Tailable cursor fails on getMore agai... Closed
is related to SERVER-33606 mongo shell startSession() should fai... Closed
is related to SERVER-37520 "too many users are authenticated" af... Closed
Backwards Compatibility: Fully Compatible
Sprint: Platforms 2018-01-01, Platforms 2018-01-15, Sharding 2018-06-04, Sharding 2018-06-18, Sharding 2018-07-02
Participants:
Linked BF Score: 0

 Description   
Existing database methods that start an implicit session

When an existing MongoDatabase method that does not take a session is called, the driver MUST check whether the deployment supports sessions (See How to Check Whether a Deployment Supports Session). If sessions are supported, the driver MUST behave as if a new ClientSession was started just for this one operation and ended immediately after this operation completes. The actual implementation will likely involve calling client.startSession, but that is not required by this spec.

Existing collection methods that start an implicit session

When an existing MongoCollection method that does not take a session is called, the driver MUST check whether the deployment supports sessions (See How to Check Whether a Deployment Supports Session). If sessions are supported, the driver MUST behave as if a new ClientSession was started just for this one operation and ended immediately after this operation completes. The actual implementation will likely involve calling client.startSession, but that is not required by this spec.

Sending the session ID to the server on all commands

When connected to a server that supports sessions a driver MUST append the session ID to every command it sends to the server (with the exceptions noted in the following section). It does this by adding a top level lsid field to the command sent to the server. A driver MUST do this without modifying any data supplied by the application (e.g. the command document passed to runCommand).:

{ commandName: ..., lsid : { id : <UUID> } }

https://github.com/mongodb/specifications/blob/cf25e1d6afe0987d8059615c2a58a518b273be66/source/sessions/driver-sessions.rst



 Comments   
Comment by Daniel Smedegaard Buus [X] [ 10/Oct/18 ]

Hi guys

I'm here via this ticket.

I've been reading this ticket and the driver sessions specification linked to in it, and I don't see how I can apply this info to overcome my issues in a shell (script) when trying to operate on a sibling db that I authenticated against.

In the specification, it instructs on starting sessions and passing along their ids to commands, but the syntax exemplified there is one fitting db.runCommand(). I can't figure out how I would go about doing say, myOtherDb.users.findOne() and passing along the session id?

Maybe it's just that this behaviour is unsupported going forward?

Thanks

Comment by Githook User [ 28/Sep/18 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit 8817328f87564a29e9be2ed1a746cf40e89587eb)

SERVER-32064 Disable implicit sessions when checking db version during connect()

(cherry picked from commit 8c5002b06fd737f75941a73785ce75e3ca7f5ce1)
Branch: v3.6
https://github.com/mongodb/mongo/commit/2e98a18d2122a30549e089bf30b96ac0c3bbf85d

Comment by Githook User [ 28/Sep/18 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit ac59bbf1d0f7c8bfbe949de8f900ffe8b835d9cb)
Branch: v3.6
https://github.com/10gen/mongo-enterprise-modules/commit/e7fe8db0c0c6f5c34c32a044cd30b78327013ec9

Comment by Githook User [ 19/Jul/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Disable implicit sessions when checking db version during connect()

(cherry picked from commit 8c5002b06fd737f75941a73785ce75e3ca7f5ce1)
Branch: v4.0
https://github.com/mongodb/mongo/commit/f81ffba33c4253901ed034e7e338fe56ef955c5b

Comment by Githook User [ 19/Jul/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit 8817328f87564a29e9be2ed1a746cf40e89587eb)
Branch: v4.0
https://github.com/mongodb/mongo/commit/493a8a144d2306d1667a0abe6bdbbad54b8c0f32

Comment by Githook User [ 19/Jul/18 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit ac59bbf1d0f7c8bfbe949de8f900ffe8b835d9cb)
Branch: v4.0
https://github.com/10gen/mongo-enterprise-modules/commit/07b6b5801da304fe761b1a85fb0f6b7f9595de52

Comment by Githook User [ 25/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Disable implicit sessions when checking db version during connect()
Branch: master
https://github.com/mongodb/mongo/commit/8c5002b06fd737f75941a73785ce75e3ca7f5ce1

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: Revert "SERVER-32064 Requests from the shell should use an implicit session by default"

This reverts commit 3225e5f0784b8306c700a6280bbb986634129291.
Branch: v4.0
https://github.com/mongodb/mongo/commit/c8f73cf4e32044d465987b0644014e92742d73f9

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: Revert "SERVER-32064 Requests from the shell should use an implicit session by default"

This reverts commit c31687bfcdf25fd9f3a6eb9a2fb07de394de873f.
Branch: v4.0
https://github.com/10gen/mongo-enterprise-modules/commit/785a1fd7c8d32948bac37eef4da575907441859e

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit 8817328f87564a29e9be2ed1a746cf40e89587eb)
Branch: v4.0
https://github.com/mongodb/mongo/commit/3225e5f0784b8306c700a6280bbb986634129291

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default

(cherry picked from commit ac59bbf1d0f7c8bfbe949de8f900ffe8b835d9cb)
Branch: v4.0
https://github.com/10gen/mongo-enterprise-modules/commit/c31687bfcdf25fd9f3a6eb9a2fb07de394de873f

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default
Branch: master
https://github.com/mongodb/mongo/commit/8817328f87564a29e9be2ed1a746cf40e89587eb

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}

Message: SERVER-32064 Requests from the shell should use an implicit session by default
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/ac59bbf1d0f7c8bfbe949de8f900ffe8b835d9cb

Comment by Jeffrey Yemin [ 30/Mar/18 ]

This might have caught, for example, SERVER-34204.

Comment by A. Jesse Jiryu Davis [ 02/Mar/18 ]

Yeah, don't do option 2. =) Don't generate a new lsid for each operation.

The spec distinguishes between ClientSessions (user-visible API objects) and ServerSessions (logical session ids or "lsids"). When the spec says that implicit sessions are "ended immediately after" the operation it doesn't mean that the actual lsid is tossed, just that the ClientSession is ended. The lsid is returned to the pool.

In drivers, when we need an lsid for an implicit session, we pull it from a pool, use it until the operation is complete, then we return it to the pool. This minimizes the number of distinct lsids we use. If you execute a command with an implicit session, its lsid is returned to the pool as soon as the command completes. If you execute "find", "aggregate", etc., then the cursor you create owns the implicit session, and returns it to the pool as soon as it receives the final batch.

The session pool is specified here, the summary is: the lsid pool is LIFO, and lsids that haven't been used in over 29 minutes are tossed, instead of used, to avoid running into the server's 30-minute idle timeout.

Comment by Kevin Pulo [ 02/Mar/18 ]

In the meantime, users looking for this functionality can workaround this by adding the following to their .mongorc.js file:

db = db.getMongo().startSession().getDatabase(db.getName());
print(db.getSession());

Or use the code at https://github.com/devkev/mongorcd/blob/master/.mongorc.d/91-default-logical-session.js (UNSUPPORTED).

Comment by Kevin Pulo [ 02/Mar/18 ]

Won't option 2 cause a profusion of logical sessions on the server? Surely creating an implicit logical session for each operation must be against the spec?

Either way, my vote is for option 1, since it feels like it better supports the use case of DBAs doing adhoc operations, which may then need to be cluster-wide killed if they go awry. I would say that the interactive shell should also print out the session id at startup, so that DBAs are aware of it.

Comment by Kaloian Manassiev [ 15/Feb/18 ]

So just to be clear - this ticket is a request to make the driver in the shell in line with the driver specification, right?

Comment by A. Jesse Jiryu Davis [ 07/Feb/18 ]

max.hirschhorn asked me to clarify how drivers handle sessions and causal consistency.

The Drivers spec for Causal Consistency requires that an explicit session be causally consistent by default:

An application requests causal consistency by creating a ClientSession with options that specify that causal consistency is desired. An application then passes the session as an argument to methods in the MongoDatabase and MongoCollection classes. Any operations performed against that session will then be causally consistent.

If the user doesn't pass a ClientSession to an operation, then the driver passes an implicit "lsid" to the server, but that operation is not ever causally consistent.

IOW, drivers use lsid two ways:

  • For causal consistency. The user creates a ClientSession object and passes it to each operation.
  • For cluster-wide killOp. The user does not create any ClientSession object, but the driver passes an "lsid" with each operation so that a DBA can kill a sharded operation by killing its session. This operation is not causally consistent because that could introduce latency, and the operation is not explicitly related to any previous operation. It could use an lsid from the driver's lsid pool, but it shouldn't be causally related to whatever operation happened to use that lsid previously.

So the specs imply (without explaining very clearly) that "explicit" sessions are causal by default, and "implicit" sessions are never causal.

Comment by Gregory McKeon (Inactive) [ 05/Feb/18 ]

Sending to sharding as we believe they own logical sessions.

Comment by Max Hirschhorn [ 21/Nov/17 ]

It would be fairly time-consuming to go through all of the methods on DB and DBCollection to implicitly create a logical session id if DB.prototype.getSession() returns a _DummyDriverSession instance. I have two proposals of how we could address this:

Option 1

Use a single session to any user operations run through a DB instance that wasn't created via a call to Mongo.prototype.startSession(). This doesn't satisfy the "and ended immediately after this operation completes" part of the specification.

diff --git a/src/mongo/shell/mongo.js b/src/mongo/shell/mongo.js
index a5077fe..3b3a2cf 100644
--- a/src/mongo/shell/mongo.js
+++ b/src/mongo/shell/mongo.js
@@ -422,11 +422,19 @@ Mongo.prototype.startSession = function startSession(options) {
 };
 
 Mongo.prototype._getDefaultSession = function getDefaultSession() {
-    // We implicitly associate a Mongo connection object with a DriverSession so that tests which
-    // call DB.prototype.getMongo() and then Mongo.prototype.getDB() to get a different DB instance
-    // are still causally consistent.
     if (!this.hasOwnProperty("_defaultSession")) {
-        this._defaultSession = new _DummyDriverSession(this);
+        // We use the existence of the TestData object to distinguish between when the mongo shell
+        // is being used for testing.
+        if (typeof TestData !== "undefined") {
+            // We implicitly associate a Mongo connection object with a DriverSession so that tests
+            // which call DB.prototype.getMongo() and then Mongo.prototype.getDB() to get a
+            // different DB instance are still causally consistent.
+            this._defaultSession = new _DummyDriverSession(this);
+        } else {
+            // We start a session when running as the user-facing mongo shell to ensure that all
+            // operations sent to the server by this client include a logical session id.
+            this._defaultSession = this.startSession();
+        }
     }
     return this._defaultSession;
 };

Option 2

Generate a new UUID for every command request.

diff --git a/src/mongo/shell/session.js b/src/mongo/shell/session.js
index 48e3eab..87e67dd 100644
--- a/src/mongo/shell/session.js
+++ b/src/mongo/shell/session.js
@@ -655,6 +655,31 @@ var {
                       client: new SessionAwareClient(client),
 
                       injectSessionId: function injectSessionId(cmdObj) {
+                          // We use the existence of the TestData object to distinguish between when
+                          // the mongo shell is being used for testing.
+                          if (typeof TestData !== "undefined") {
+                              return cmdObj;
+                          }
+
+                          // We generate a fresh logic session id when running as the user-facing
+                          // mongo shell to ensure that all operations sent to the server by this
+                          // client include a logical session id.
+                          cmdObj = Object.assign({}, cmdObj);
+
+                          const cmdName = Object.keys(cmdObj)[0];
+
+                          // If the command is in a wrapped form, then we look for the actual
+                          // command object inside the query/$query object.
+                          let cmdObjUnwrapped = cmdObj;
+                          if (cmdName === "query" || cmdName === "$query") {
+                              cmdObj[cmdName] = Object.assign({}, cmdObj[cmdName]);
+                              cmdObjUnwrapped = cmdObj[cmdName];
+                          }
+
+                          if (!cmdObjUnwrapped.hasOwnProperty("lsid")) {
+                              cmdObjUnwrapped.lsid = {id: UUID()};
+                          }
+
                           return cmdObj;
                       },

Generated at Thu Feb 08 04:29:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.