Details
Description
Very large Bulk batched write operations inside a transaction fail with transaction conflicts.
Error from the reproduction script:
Creating operations
|
Starting transaction with 200 operations |
Start session 25e4510e-da67-47c6-9634-4d28242ca3ab |
Synthetic op to start transaction
|
Attempting bulk write of 200 operations |
Inside transaction 1 |
Synthetic op to start transaction
|
Attempting bulk write of 200 operations |
Inside transaction 2 |
Synthetic op to start transaction
|
...
|
Attempting bulk write of 200 operations |
Inside transaction 164 |
Traceback (most recent call last):
|
File "bulk_transactions.py", line 84, in <module> |
session.with_transaction(
|
File "/usr/local/lib/python3.9/site-packages/pymongo/client_session.py", line 620, in with_transaction |
ret = callback(self) |
File "bulk_transactions.py", line 72, in perform_operation |
res = coll.bulk_write(ops, ordered=False, session=s) |
File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 508, in bulk_write |
bulk_api_result = blk.execute(write_concern, session) |
File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 529, in execute |
return self.execute_command(generator, write_concern, session) |
File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 360, in execute_command |
client._retry_with_session(
|
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1438, in _retry_with_session |
return self._retry_internal(retryable, func, session, bulk) |
File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1470, in _retry_internal |
return func(session, sock_info, retryable) |
File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 354, in retryable_bulk |
self._execute_command( |
File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 310, in _execute_command |
result, to_send = bwc.execute(ops, client) |
File "/usr/local/lib/python3.9/site-packages/pymongo/message.py", line 950, in execute |
result = self.write_command(request_id, msg, to_send) |
File "/usr/local/lib/python3.9/site-packages/pymongo/message.py", line 1042, in write_command |
reply = self.sock_info.write_command(request_id, msg) |
File "/usr/local/lib/python3.9/site-packages/pymongo/pool.py", line 798, in write_command |
helpers._check_command_response(result, self.max_wire_version) |
File "/usr/local/lib/python3.9/site-packages/pymongo/helpers.py", line 167, in _check_command_response |
raise OperationFailure(errmsg, code, response, max_wire_version) |
pymongo.errors.OperationFailure: Encountered error from localhost:27018 during a transaction :: caused by :: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction., full error: {'ok': 0.0, 'errmsg': 'Encountered error from localhost:27018 during a transaction :: caused by :: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.', 'code': 112, 'codeName': 'WriteConflict', 'operationTime': Timestamp(1632204487, 4), '$clusterTime': {'clusterTime': Timestamp(1632204487, 4), 'signature': {'hash': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'keyId': 0}}, 'errorLabels': ['TransientTransactionError']} |
Python version: 3.9.7
pymongo version: 3.12.0
mongod version: 4.4.6-ent (sharded)
Attachments
Issue Links
- related to
-
PYTHON-2915 Session persistence failure under transaction callback with large Bulk ops
-
- Closed
-