-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Docs
I ran some simple tests with pymongo and transactions. Consider the following 4 sequences/transactions:
Common code:
amount = random.randint(-100, 100)
result1 = db.hltest.find_one( { '_id': 1 }, session=session )
db.hltest.update_one( {'_id': 1}, {'$inc': {'n': -amount}}, session=session )
result2 = db.hltest.find_one( { '_id': 2 }, session=session )
db.hltest.update_one( {'_id': 2}, {'$inc': {'n': amount}}, session=session )
cursor = db.hltest.aggregate( [ { '$group': { '_id': 'foo', 'total' : { '$sum': '$n' } } } ], session=session )
result = cursor.next()
if result['total'] != 200:
print "ERROR: sum total = %s" % result['total']
Case 1:
session=None write_concern=WriteConcern(w=1) read_concern=ReadConcern(level="local") client = pymongo.MongoClient( config["mongodb_url"] ) db = pymongo.database.Database(client=client, name="test", write_concern=write_concern, read_concern=read_concern) # Common code...
Case 2:
session=None write_concern=WriteConcern(w="majority") read_concern=ReadConcern(level="majority") client = pymongo.MongoClient( config["mongodb_url"] ) db = pymongo.database.Database(client=client, name="test", write_concern=write_concern, read_concern=read_concern) # Common code...
Case 3:
write_concern=WriteConcern(w="majority") read_concern=ReadConcern(level="snapshot") client = pymongo.MongoClient( config["mongodb_url"] ) db = pymongo.database.Database(client=client, name="test", write_concern=write_concern, read_concern=read_concern) with db.client.start_session(causal_consistency=True) as session: session.start_transaction() # Common code... session.commit_transaction()
Case 4:
write_concern=WriteConcern(w="majority") read_concern=ReadConcern(level="snapshot") client = pymongo.MongoClient( config["mongodb_url"] ) db = pymongo.database.Database(client=client, name="test", write_concern=write_concern, read_concern=read_concern) with db.client.start_session(causal_consistency=True) as session: session.start_transaction(write_concern=write_concern, read_concern=read_concern) # Common code... session.commit_transaction()
As I was measuring latencies, I notice that the updates for Case 1 and Case 3 take the same (short) amount of time and Case 2 and Case 4 take the same (longer) amount of time. This indicates that Case 3 is really using w:1, even if I intended to use w:majority.
There is no exception raised or other indication that I'm doing something wrong. As it happens, mongod.log slow query output also doesn't log the writeConcern or readConcern, even if it does mention readPreference.
Update:
Consider also Case 5, where a session is used, but no transaction:
write_concern=WriteConcern(w="majority") read_concern=ReadConcern(level="majority") client = pymongo.MongoClient( config["mongodb_url"] ) db = pymongo.database.Database(client=client, name="test", write_concern=write_concern, read_concern=read_concern) with db.client.start_session(causal_consistency=True) as session: # Common code...
Now, the mongod.log entries (with setProfilingLevel(1, -1) for
Case 5, using session, write-and-read-concerns inherited from db object:
2018-11-07T07:46:41.307+0200 I COMMAND [conn38] command test.$cmd command: update { update: "hltest", ordered: true, writeConcern: { w: "majority" }, lsid: { id: UUID("2826488c-c490-4a85-a28f-b3826592c173") }, txnNumber: 1, $clusterTime: { clusterTime: Timestamp(1541569601, 4), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "test", $readPreference: { mode: "primary" } } numYields:0 reslen:245 locks:{ Global: { acquireCount: { r: 5, w: 3 } }, Database: { acquireCount: { r: 1, w: 4 } }, Collection: { acquireCount: { r: 1, w: 3 } }, Metadata: { acquireCount: { W: 1 } }, oplog: { acquireCount: { w: 1 } } } protocol:op_msg 2ms
2018-11-07T07:46:41.308+0200 I COMMAND [conn38] command test.hltest command: find { find: "hltest", filter: { _id: 2 }, limit: 1, singleBatch: true, readConcern: { afterClusterTime: Timestamp(1541569601, 5), level: "majority" }, lsid: { id: UUID("2826488c-c490-4a85-a28f-b3826592c173") }, $clusterTime: { clusterTime: Timestamp(1541569601, 5), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "test", $readPreference: { mode: "primaryPreferred" } } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturned:1 reslen:249 locks:{ Global: { acquireCount: { r: 1 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_msg 0ms
Case 3, in a transaction, write-and-read-concerns not inherited:
2018-11-07T07:46:44.436+0200 I COMMAND [conn46] command test.hltest command: find { find: "hltest", filter: { _id: 1 }, limit: 1, singleBatch: true, lsid: { id: UUID("46fb776c-fc74-4a5c-a8e7-a1d861db7504") }, startTransaction: true, txnNumber: 100, autocommit: false, $clusterTime: { clusterTime: Timestamp(1541569604, 102), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "test", $readPreference: { mode: "primaryPreferred" } } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturned:1 reslen:249 locks:{} protocol:op_msg 0ms
...
2018-11-07T07:46:44.440+0200 I COMMAND [conn46] command admin.$cmd command: commitTransaction { commitTransaction: 1, txnNumber: 100, autocommit: false, lsid: { id: UUID("46fb776c-fc74-4a5c-a8e7-a1d861db7504") }, $clusterTime: { clusterTime: Timestamp(1541569604, 102), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "admin", $readPreference: { mode: "primary" } } numYields:0 reslen:163 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 3 } }, Collection: { acquireCount: { w: 2 } }, oplog: { acquireCount: { w: 1 } } } protocol:op_msg 0ms
Case 4, write-and-read-concern set in start_transaction():
2018-11-07T07:46:45.411+0200 I COMMAND [conn50] command test.hltest command: find { find: "hltest", filter: { _id: 1 }, limit: 1, singleBatch: true, lsid: { id: UUID("a2d9e096-02a3-41ee-88f2-a8af76f5ee00") }, startTransaction: true, readConcern: { level: "snapshot" }, txnNumber: 100, autocommit: false, $clusterTime: { clusterTime: Timestamp(1541569605, 59), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "test", $readPreference: { mode: "primaryPreferred" } } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturned:1 reslen:249 locks:{} protocol:op_msg 0ms
...
2018-11-07T07:46:45.419+0200 I COMMAND [conn50] command admin.$cmd command: commitTransaction { commitTransaction: 1, txnNumber: 100, autocommit: false, writeConcern: { w: "majority" }, lsid: { id: UUID("a2d9e096-02a3-41ee-88f2-a8af76f5ee00") }, $clusterTime: { clusterTime: Timestamp(1541569605, 59), signature: { keyId: 0, hash: BinData(0, 0000000000000000000000000000000000000000) } }, $db: "admin", $readPreference: { mode: "primary" } } numYields:0 reslen:163 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 3 } }, Collection: { acquireCount: { w: 2 } }, oplog: { acquireCount: { w: 1 } } } protocol:op_msg 4ms
To summarize the reason for this bug report:
- As discussed in comments, the spec didn't fully make it clear to me what I should expect about write-and-read-concern inherited into a transaction.
- Regardless of spec, it doesn't make sense to me - and looks dangerous from an end user perspective - that db.coll.update(..., session=s) will use the writeConcern from db if it's just a session but will silently ignore it if s.start_transaction() was called previously. It's the exact same command, so same rules should apply.
- is related to
-
-
- Closed
-
-
DRIVERS-2172 db and coll handles write concern and read concern are silently ignored in transactions
-
- Closed
-
- related to
-
PYTHON-1508 Implement Transactions spec
-
- Closed
-