Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Done
-
4.1.11
-
None
-
None
-
None
-
ALL
-
-
Storage NYC 2019-05-20, Sharding 2019-05-06
-
45
Description
Various driver transaction spec tests are failing on latest sharded clusters. The one commonality between the failures is that they all use read concern "snapshot" and they all fail with SnapshotTooOld on the first operation in the transaction. Was there a recent change in how "snapshot" works?
Although this issue fails consistently in Evergreen, creating a standalone repro has proven difficult. When starting a fresh sharded cluster the attached repro (PYTHON-1796.py) succeeds. However if I run the entire driver test suite against the cluster first, then the repro starts failing consistently with SnapshotTooOld. Here's an example evergreen failure from today:
https://evergreen.mongodb.com/task/mongo_python_driver_tests_python_version_supports_openssl_110_test_ssl__auth_ssl~auth_ssl_python_version_requires_openssl_102_plus~3.7_test_latest_sharded_cluster_ea8941ef5d3f60a227cb89021ef7d65d7b06c6e1_19_04_16_21_00_56
And here's the repro failing:
$ python PYTHON-1796.py
|
First transaction command failed: {'ok': 0.0, 'errmsg': 'Transaction 060cae38-7be3-4240-babe-65739200884c:1 was aborted on statement 0 due to: a non-retryable snapshot error :: caused by :: Encountered error from localhost:27019 during a transaction :: caused by :: Read timestamp Timestamp(1555453221, 10) is older than the oldest available timestamp.', 'code': 239, 'codeName': 'SnapshotTooOld', 'operationTime': Timestamp(1555453221, 11), '$clusterTime': {'clusterTime': Timestamp(1555453221, 11), 'signature': {'hash': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'keyId': 0}}, 'errorLabels': ['TransientTransactionError']}
|
Traceback (most recent call last):
|
File "PYTHON-1796.py", line 13, in <module>
|
coll.insert_one({'_id': i}, session=s)
|
File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 694, in insert_one
|
session=session),
|
File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 608, in _insert
|
bypass_doc_val, session)
|
File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 596, in _insert_one
|
acknowledged, _insert_command, session)
|
File "/home/ubuntu/mongo-python-driver/pymongo/mongo_client.py", line 1342, in _retryable_write
|
return self._retry_with_session(retryable, func, s, None)
|
File "/home/ubuntu/mongo-python-driver/pymongo/mongo_client.py", line 1295, in _retry_with_session
|
return func(session, sock_info, retryable)
|
File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 591, in _insert_command
|
retryable_write=retryable_write)
|
File "/home/ubuntu/mongo-python-driver/pymongo/pool.py", line 579, in command
|
unacknowledged=unacknowledged)
|
File "/home/ubuntu/mongo-python-driver/pymongo/network.py", line 150, in command
|
parse_write_concern_error=parse_write_concern_error)
|
File "/home/ubuntu/mongo-python-driver/pymongo/helpers.py", line 155, in _check_command_response
|
raise OperationFailure(msg % errmsg, code, response)
|
pymongo.errors.OperationFailure: Transaction 060cae38-7be3-4240-babe-65739200884c:1 was aborted on statement 0 due to: a non-retryable snapshot error :: caused by :: Encountered error from localhost:27019 during a transaction :: caused by :: Read timestamp Timestamp(1555453221, 10) is older than the oldest available timestamp.
|
I can reproduce this on the latest version:
db version v4.1.10-94-ga654dcf
git version: a654dcf592ea7ed65426a0de96b4079ff4fc6716
Attachments
Issue Links
- causes
-
PYTHON-1796 Transaction tests that use readConcern "snapshot" are failing with snapshot errors
-
- Closed
-
- depends on
-
SERVER-36956 Replace the statistic that dynamically resizes the snapshot history window
-
- Closed
-
- is depended on by
-
CSHARP-2629 Remove workaround for SERVER-40685
-
- Closed
-
- is related to
-
PYTHON-1901 Avoid using readConcern snapshot in sharded transaction tests
-
- Closed
-
-
CSHARP-2601 Fix consistently failing convenient api for transactions test on latest
-
- Closed
-
-
SERVER-41472 Create a workload to demonstrate the MongoDB usage difficulties of the WT WT_STAT_CONN_CACHE_LOOKASIDE_SCORE stat
-
- Closed
-
-
RUBY-1942 Avoid using readConcern snapshot in sharded transaction tests
-
- Closed
-
- related to
-
CDRIVER-3301 Unskip txn tests with readConcern=snapshot against mongos
-
- Closed
-