The CSOT rtt heuristic is incompatible with loadBalanced=True mode:
[2022/07/04 20:47:34.387] ERROR [0.216s]: test_maxTimeMS_value_in_the_command_is_less_than_timeoutMS (test_csot.TestCsotCommandExecution) [2022/07/04 20:47:34.387] ---------------------------------------------------------------------- [2022/07/04 20:47:34.387] Traceback (most recent call last): [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1472, in test_case [2022/07/04 20:47:34.387] self.run_scenario(spec) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1449, in run_scenario [2022/07/04 20:47:34.387] self.run_operations(spec["operations"]) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1376, in run_operations [2022/07/04 20:47:34.387] self.run_entity_operation(op) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1199, in run_entity_operation [2022/07/04 20:47:34.387] result = cmd(**dict(arguments)) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 626, in insert_one [2022/07/04 20:47:34.387] self._insert_one( [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 567, in _insert_one [2022/07/04 20:47:34.387] self.__database.client._retryable_write(acknowledged, _insert_command, session) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1433, in _retryable_write [2022/07/04 20:47:34.387] return self._retry_with_session(retryable, func, s, None) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1318, in _retry_with_session [2022/07/04 20:47:34.387] return self._retry_internal(retryable, func, session, bulk) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1353, in _retry_internal [2022/07/04 20:47:34.387] return func(session, sock_info, retryable) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 555, in _insert_command [2022/07/04 20:47:34.387] result = sock_info.command( [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/pool.py", line 793, in command [2022/07/04 20:47:34.387] self._raise_connection_failure(error) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/pool.py", line 765, in command [2022/07/04 20:47:34.387] return command( [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/network.py", line 121, in command [2022/07/04 20:47:34.387] sock_info.apply_timeout(client, spec, write_concern) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/pool.py", line 583, in apply_timeout [2022/07/04 20:47:34.387] max_time_ms = timeout - rtt [2022/07/04 20:47:34.387] TypeError: unsupported operand type(s) for -: 'float' and 'NoneType' [2022/07/04 20:47:34.387] ====================================================================== [2022/07/04 20:47:34.387] FAIL [0.057s]: test_timeoutMS_applied_to_entire_bulkWrite,_not_individual_commands (test_csot.TestCsotBulkWrite) [2022/07/04 20:47:34.387] ---------------------------------------------------------------------- [2022/07/04 20:47:34.387] Traceback (most recent call last): [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1199, in run_entity_operation [2022/07/04 20:47:34.387] result = cmd(**dict(arguments)) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 534, in bulk_write [2022/07/04 20:47:34.387] bulk_api_result = blk.execute(write_concern, session) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/bulk.py", line 514, in execute [2022/07/04 20:47:34.387] return self.execute_command(generator, write_concern, session) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/bulk.py", line 392, in execute_command [2022/07/04 20:47:34.387] client._retry_with_session(self.is_retryable, retryable_bulk, s, self) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1318, in _retry_with_session [2022/07/04 20:47:34.387] return self._retry_internal(retryable, func, session, bulk) [2022/07/04 20:47:34.387] TypeError: unsupported operand type(s) for -: 'float' and 'NoneType' [2022/07/04 20:47:34.387] During handling of the above exception, another exception occurred: [2022/07/04 20:47:34.387] Traceback (most recent call last): [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1472, in test_case [2022/07/04 20:47:34.387] self.run_scenario(spec) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1449, in run_scenario [2022/07/04 20:47:34.387] self.run_operations(spec["operations"]) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1376, in run_operations [2022/07/04 20:47:34.387] self.run_entity_operation(op) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1208, in run_entity_operation [2022/07/04 20:47:34.387] return self.process_error(exc, expect_error) [2022/07/04 20:47:34.387] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 955, in process_error [2022/07/04 20:47:34.388] self.assertIsInstance( [2022/07/04 20:47:34.388] AssertionError: TypeError("unsupported operand type(s) for -: 'float' and 'NoneType'") is not an instance of (<class 'pymongo.errors.NetworkTimeout'>, <class 'pymongo.errors.ExecutionTimeout'>, <class 'pymongo.errors.ServerSelectionTimeoutError'>) [2022/07/04 20:47:34.388] ====================================================================== [2022/07/04 20:47:34.388] FAIL [0.219s]: test_command_is_not_sent_if_RTT_is_greater_than_timeoutMS (test_csot.TestCsotCommandExecution) [2022/07/04 20:47:34.388] ---------------------------------------------------------------------- [2022/07/04 20:47:34.388] Traceback (most recent call last): [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1199, in run_entity_operation [2022/07/04 20:47:34.388] result = cmd(**dict(arguments)) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 626, in insert_one [2022/07/04 20:47:34.388] self._insert_one( [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/collection.py", line 567, in _insert_one [2022/07/04 20:47:34.388] self.__database.client._retryable_write(acknowledged, _insert_command, session) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1433, in _retryable_write [2022/07/04 20:47:34.388] return self._retry_with_session(retryable, func, s, None) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/pymongo/mongo_client.py", line 1318, in _retry_with_session [2022/07/04 20:47:34.388] return self._retry_internal(retryable, func, session, bulk) [2022/07/04 20:47:34.388] TypeError: unsupported operand type(s) for -: 'float' and 'NoneType' [2022/07/04 20:47:34.388] During handling of the above exception, another exception occurred: [2022/07/04 20:47:34.388] Traceback (most recent call last): [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1472, in test_case [2022/07/04 20:47:34.388] self.run_scenario(spec) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1449, in run_scenario [2022/07/04 20:47:34.388] self.run_operations(spec["operations"]) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1376, in run_operations [2022/07/04 20:47:34.388] self.run_entity_operation(op) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 1208, in run_entity_operation [2022/07/04 20:47:34.388] return self.process_error(exc, expect_error) [2022/07/04 20:47:34.388] File "/data/mci/7e45ef05a52098a087c4d4c579126acd/src/test/unified_format.py", line 955, in process_error [2022/07/04 20:47:34.388] self.assertIsInstance( [2022/07/04 20:47:34.388] AssertionError: TypeError("unsupported operand type(s) for -: 'float' and 'NoneType'") is not an instance of (<class 'pymongo.errors.NetworkTimeout'>, <class 'pymongo.errors.ExecutionTimeout'>, <class 'pymongo.errors.ServerSelectionTimeoutError'>)
The problem is that load balanced mode does not run SDAM and hence does not have an rtt at all. Some ways to fix this:
- use the rtt from the connection handshake for the lifetime of the connection.
- use a fixed value for the rtt, eg 100ms