Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-857

Wire protocol race with mixed versions of mongos

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Fully Compatible

      Mongos load balancing (PYTHON-852) introduces a race related to MongoClient._writable_max_wire_version. If I start a sharded cluster with a 2.4 and a 2.6 mongos and connect to both, half the time _writable_max_wire_version is 2, half the time it is zero.

      After a small random number of iterations this code:

      for i in range(1000):
          client = MongoClient('mongodb://localhost:1026,localhost:1027')
          print i

      ... attempts a 2.6-style write command against the 2.4 mongos and throws:

      Traceback (most recent call last):
        File "writable.py", line 6, in <module>
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/collection.py", line 1921, in insert
          check_keys, manipulate, write_concern)
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/collection.py", line 411, in _insert
          gen(), check_keys, self.codec_options, client)
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/message.py", line 435, in _do_batched_write_command
          results.append((idx_offset, send_message()))
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/message.py", line 387, in send_message
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/mongo_client.py", line 804, in _send_message
          return self.__check_gle_response(response.data, command)
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/mongo_client.py", line 727, in __check_gle_response
        File "/Users/emptysquare/.virtualenvs/official/mongo-python-driver/pymongo/helpers.py", line 179, in _check_command_response
          raise OperationFailure(msg % errmsg, code, response)
      pymongo.errors.OperationFailure: no such cmd: insert

      We can use this opportunity to remove races related to outdated ServerDescriptions, and rely on each socket's ismaster response (PYTHON-829):

      • *DONE*: Remove MongoClient._writable_max_wire_version().
      • *DONE*: Methods that used it must select a server and call get_socket() for a SocketInfo, and use that SocketInfo's wire version (which came from an ismaster call on that socket when it was opened) to determine the wire protocol, before formatting a message.
      • *DONE*: Functions in message.py (and _cmessagemodule) that took a client instance take SocketInfo instead.
      • *DONE*: Add SocketInfo.send_message_with_response and other APIs that message.py needs.
      • *DONE*: Move Server._check_bson_size to SocketInfo, ensure it's tested for all paths.
      • *DONE*: Refactor SocketInfo.command, helpers._command, and network.command?
      • *DONE*: Delete MongoClient._send_message?
      • *DONE*: Actually retest against live mixed version cluster, and add test to pymongo-mockup-tests
      • *DONE*: For absolute correctness in Collection.options(), and Database.collection_names() we should get a socket, check its wire version, and if it's < 3 use that socket to query the system collection
      • *DONE*: Delete test_wire_version_mongos_ha, which should have been deleted in PYTHON-852 anyway.
      • *DONE*: Delete max_wire_version etc. from MongoClient

            bernie@mongodb.com Bernie Hackett
            jesse@mongodb.com A. Jesse Jiryu Davis
            0 Vote for this issue
            1 Start watching this issue