[SERVER-8384] Using $set with upsert against sharded cluster behaves differently to non-sharded cluster Created: 29/Jan/13  Updated: 11/Jul/16  Resolved: 15/Dec/13

Status: Closed
Project: Core Server
Component/s: Sharding, Write Ops
Affects Version/s: 2.2.1
Fix Version/s: 2.5.5

Type: Bug Priority: Major - P3
Reporter: James Blackburn Assignee: Scott Hernandez (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

RHEL 5


Issue Links:
Duplicate
duplicates SERVER-9074 Upsert fails with "cannot modify shar... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

On a shared collection, sharded on 'symbol' you get:

document = {'symbol': 'random_symbol', 'value': 'val'}
col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True)
Traceback (most recent call last):
  File "/users/is/jblackburn/pyenvs/research/lib/python2.6/site-packages/ipython-0.11_5-py2.6.egg/IPython/core/interactiveshell.py", line 2400, in run_code
    exec code_obj in self.user_global_ns, self.user_ns
  File "<ipython-input-50-74f5bdbf4070>", line 1, in <module>
    col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/collection.py", line 481, in update
    check_keys, self.__uuid_subtype), safe)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 852, in _send_message
    rv = self.__check_response_to_last_error(response)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 795, in __check_response_to_last_error
    raise OperationFailure(details["err"], details["code"])
OperationFailure: Can't modify shard key's value. field: symbol: "random_symbol" collection: mongoose_jblackburn.centaur

On an unsharded collection, you get:

col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True)
Out[54]: 
{u'connectionId': 63367,
 u'err': None,
 u'lastOp': Timestamp(1359483484, 1),
 u'n': 1,
 u'ok': 1.0,
 u'singleShard': u'rs0/cn14-ib:27118,cn17-ib:27118,cn53-ib:27118',
 u'updatedExisting': False,
 u'upserted': ObjectId('5108125c5fa01f07fcb07c5f')}

Participants:

 Description   

If you do an update with upsert on a sharded cluster, and the update document contains only update operators, but the '$set' update also includes the shard key, you get an error even if the key-value hasn't changed:

    exec code_obj in self.user_global_ns, self.user_ns
  File "<ipython-input-50-74f5bdbf4070>", line 1, in <module>
    col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/collection.py", line 481, in update
    check_keys, self.__uuid_subtype), safe)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 852, in _send_message
    rv = self.__check_response_to_last_error(response)
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 795, in __check_response_to_last_error
    raise OperationFailure(details["err"], details["code"])
OperationFailure: Can't modify shard key's value. field: symbol: "random_symbol" collection: mongoose_jblackburn.centaur

The bad thing is that the code works fine on an unsharded collection, and only breaks when the collection is sharded - making it difficult to test.



 Comments   
Comment by Scott Hernandez (Inactive) [ 15/Dec/13 ]

This has been fixed with changes like SERVER-9074, and full support for testing from old clients in other issues.

Tests covering this case are in: jstests/sharding/shard_key_immutable.js

Comment by James Blackburn [ 15/Feb/13 ]

We saw this again in another code path

It's quite hard to test for behavioural differences between a sharded and an unsharded mongod. Especially, as in this case, the update hasn't attempted to change the field.

In Pymongo:

 
document = {'symbol': 'random_symbol', 'value': 'val'} 
col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True) 
Traceback (most recent call last): 
  File "/users/is/jblackburn/pyenvs/research/lib/python2.6/site-packages/ipython-0.11_5-py2.6.egg/IPython/core/interactiveshell.py", line 2400, in run_code 
    exec code_obj in self.user_global_ns, self.user_ns 
  File "<ipython-input-50-74f5bdbf4070>", line 1, in <module> 
    col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True) 
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/collection.py", line 481, in update 
    check_keys, self.__uuid_subtype), safe) 
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 852, in _send_message 
    rv = self.__check_response_to_last_error(response) 
  File "/local/home/jblackburn/net/pymongo-2.4.1/pymongo/mongo_client.py", line 795, in __check_response_to_last_error 
    raise OperationFailure(details["err"], details["code"]) 
OperationFailure: Can't modify shard key's value. field: symbol: "random_symbol" collection: mongoose_jblackburn.centaur 

On an unsharded collection, you get:

 
col.update({'symbol': document['symbol']}, {'$set': document}, w=1, upsert=True) 
Out[54]: 
{u'connectionId': 63367, 
 u'err': None, 
 u'lastOp': Timestamp(1359483484, 1), 
 u'n': 1, 
 u'ok': 1.0, 
 u'singleShard': u'rs0/cn14-ib:27118,cn17-ib:27118,cn53-ib:27118', 
 u'updatedExisting': False, 
 u'upserted': ObjectId('5108125c5fa01f07fcb07c5f')} 

Comment by James Blackburn [ 29/Jan/13 ]

Also worth noting, that we're not trying to change the shard-key's value here.

Generated at Thu Feb 08 03:17:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.