Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13175

Write command with invalid write concern crashes mongos in a mixed cluster.

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Blocker - P1 Blocker - P1
    • 2.6.0-rc2
    • Affects Version/s: 2.6.0-rc1
    • Component/s: Sharding
    • Labels:
    • ALL
    • Hide
      1. Set up a mixed version cluster as mentioned in the description.
      2. Shard a collection on _id.
      3. Insert enough documents into the sharded collection that chunks exist on both shards.
      4. Do a multi update using a $in query that matches documents on both shards using a write concern that not all shards support.

      In my example both shards are standalone mongod. w > 1 write concern causes mongos to crash.

      Show
      Set up a mixed version cluster as mentioned in the description. Shard a collection on _id. Insert enough documents into the sharded collection that chunks exist on both shards. Do a multi update using a $in query that matches documents on both shards using a write concern that not all shards support. In my example both shards are standalone mongod. w > 1 write concern causes mongos to crash.

      Using a mixed version cluster (2.6 mongos and config, 2.6 shard, 2.4 shard), a write operation that affects both shards causes mongos to crash if the write concern is invalid for one of the shards.

      Commands in python:

      >>> c.test.command('update', 'test', updates=[SON([('q', {'_id': {'$in': [2, 99997]}}), ('u', {'$set': {'foo': 'bar'}}), ('upsert', False), ('multi', True)])])
      {u'nModified': 1, u'ok': 1, u'n': 2}
      >>> 
      >>> c.test.command('update', 'test', updates=[SON([('q', {'_id': {'$in': [3, 99996]}}), ('u', {'$set': {'foo': 'bar'}}), ('upsert', False), ('multi', True)])], writeConcern={'w': 2, 'wtimeout': 100})
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "pymongo/database.py", line 435, in command
          uuid_subtype, compile_re, **kwargs)[0]
        File "pymongo/database.py", line 335, in _command
          for doc in cursor:
        File "pymongo/cursor.py", line 1017, in next
          if len(self.__data) or self._refresh():
        File "pymongo/cursor.py", line 961, in _refresh
          self.__uuid_subtype))
        File "pymongo/cursor.py", line 885, in __send_message
          res = client._send_message_with_response(message, **kwargs)
        File "pymongo/mongo_client.py", line 1201, in _send_message_with_response
          raise AutoReconnect(str(e))
      pymongo.errors.AutoReconnect: connection closed
      

      mongos log:

      2014-03-12T13:54:56.212-0700 [mongosMain] MongoS version 2.6.0-rc2-pre- starting: pid=92669 port=27017 64-bit host=devtop.local (--help for usage)
      2014-03-12T13:54:56.212-0700 [mongosMain] db version v2.6.0-rc2-pre-
      2014-03-12T13:54:56.213-0700 [mongosMain] git version: 27afe048247d1ca3d32858bd9047fc92e97efea8
      ...
      2014-03-12T13:55:50.499-0700 [Balancer] distributed lock 'balancer/devtop.local:27017:1394657696:16807' unlocked. 
      2014-03-12T13:55:52.067-0700 [conn1] ERROR: Uncaught std::exception: vector::_M_range_check, terminating
      2014-03-12T13:55:52.067-0700 [conn1] dbexit:  rc:100
      

      sh.status:

      mongos> sh.status()
      --- Sharding Status --- 
        sharding version: {
      	"_id" : 1,
      	"version" : 4,
      	"minCompatibleVersion" : 4,
      	"currentVersion" : 5,
      	"clusterId" : ObjectId("531fe9146e3028d5a974d0b6")
      }
        shards:
      	{  "_id" : "shard0000",  "host" : "localhost:37017" }
      	{  "_id" : "shard0001",  "host" : "localhost:37018" }
        databases:
      	{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
      	{  "_id" : "test",  "partitioned" : true,  "primary" : "shard0000" }
      		test.test
      			shard key: { "_id" : 1 }
      			chunks:
      				shard0001	3
      				shard0000	4
      			{ "_id" : { "$minKey" : 1 } } -->> { "_id" : 0 } on : shard0001 Timestamp(2, 0) 
      			{ "_id" : 0 } -->> { "_id" : 15093 } on : shard0001 Timestamp(3, 0) 
      			{ "_id" : 15093 } -->> { "_id" : 31137 } on : shard0001 Timestamp(4, 0) 
      			{ "_id" : 31137 } -->> { "_id" : 47181 } on : shard0000 Timestamp(4, 1) 
      			{ "_id" : 47181 } -->> { "_id" : 69233 } on : shard0000 Timestamp(3, 2) 
      			{ "_id" : 69233 } -->> { "_id" : 95999 } on : shard0000 Timestamp(3, 4) 
      			{ "_id" : 95999 } -->> { "_id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(3, 5) 
      

            Assignee:
            randolph@mongodb.com Randolph Tan
            Reporter:
            bernie@mongodb.com Bernie Hackett
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: