[SERVER-9074] Upsert fails with "cannot modify shard key" when not modifying shard key Created: 21/Mar/13 Updated: 11/Jul/16 Resolved: 14/Nov/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.4.0 |
| Fix Version/s: | 2.5.4 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Chad Wallace | Assignee: | Scott Hernandez (Inactive) |
| Resolution: | Done | Votes: | 6 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Debian Wheezy amd64 |
||
| Attachments: |
|
||||||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Description |
|
After upgrading to 2.4.0, updates stopped working. All of my updates are upserts, and they fail with the message "cannot modify shard key for collection tripquest.persistentlistings, found new value for q.md", both through the Perl MongoDB module and in the mongo shell. I've checked and double-checked, and the shard key is not being modified. It even happens when there is NO existing document. A straight insert does work. It also works if I run a 2.2.3 mongos and connect to the same configdb, with the same mongod's (all 2.4) behind it. With a 2.4.0 mongos, it doesn't work. Also, it complains that it found a new value for the first entry in "q". If I put "de" first, it complains that "de" is the new value. Likewise for "gw". The shard key is a composite: { "q.de" : 1, "q.gw" : 1, "q.md" : 1 } |
| Comments |
| Comment by Githook User [ 12/Dec/13 ] | |||||||||||||||||||||
|
Author: {u'username': u'acmorrow', u'name': u'Andrew Morrow', u'email': u'acm@10gen.com'}Message: | |||||||||||||||||||||
| Comment by auto [ 26/Oct/13 ] | |||||||||||||||||||||
|
Author: {u'username': u'acmorrow', u'name': u'Andrew Morrow', u'email': u'acm@10gen.com'}Message: | |||||||||||||||||||||
| Comment by Randolph Tan [ 15/Jul/13 ] | |||||||||||||||||||||
|
Hi, The okForStorage is a sanity check performed in the server for the validity of the BSON document and it was rejecting the $date field that you used in your object. To get around this, you can use the Date object to replace the $date like this:
becomes:
You can also get more information about extended json here: | |||||||||||||||||||||
| Comment by Juan Cervera [ 27/Jun/13 ] | |||||||||||||||||||||
|
Just for a test, I removed the subdocument and still produced me problems, see below:
What's this "not okForStorage" about? If I remove the $set bit and put the key back in the update area it seems to work though ... see below:
Note though that removing the subdocument is really not a palatable option as we would have to migrate a lot of production data to the new structure, something I would rather avoid. | |||||||||||||||||||||
| Comment by Juan Cervera [ 27/Jun/13 ] | |||||||||||||||||||||
|
Doug, I've tried the approach that you suggest but still gives problems if the shard key is part of a subdocument and other bits of that subdocument are being udpated, see example below:
Any suggestions for this scenario that wouldn't require extracting the key out of "content"? Thanks, | |||||||||||||||||||||
| Comment by Doug Woos [ 09/May/13 ] | |||||||||||||||||||||
|
Here is an update where the shard key is in the query but not in the update:
This makes sense, since an update does a replace and the shard key is not in the replacement dictionary. How would you modify this query so that it does not error out? | |||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 09/May/13 ] | |||||||||||||||||||||
|
Save is just a (special) update with the query of the _id. The full shard key will always need to be there for save to work, and will always be included since you doing a full document replacement in the update part. Any save can be manually converted to a regular update w/upsert-flag. You can include the shard key in the query but not the update and it will work just fine. Save is a special case which doesn't work well in a sharded cluster unless you shard by the _id field. To sum it up, save works just fine on sharded collections since it must include the full shard key. | |||||||||||||||||||||
| Comment by Doug Woos [ 09/May/13 ] | |||||||||||||||||||||
|
Scott, When doing a save (as opposed to an atomic update) you definitely need to include the shard key in the update for it to work at all--the error text if you don't even says "full shard key must be in update object for collection". And the shard key has to be in the query in order for it to be targeted, right? So as it stands it is impossible to reliably do full saves (as opposed to atomic updates) on sharded collections. | |||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 09/May/13 ] | |||||||||||||||||||||
|
The query is used as the base for the insert, if the document isn't found to update, so as long as you include the shard key in the query it will work out; you don't need to include it in both, and in some cases doing so will cause problems when depending on the form used in the update part. | |||||||||||||||||||||
| Comment by Kiril Savino [ 09/May/13 ] | |||||||||||||||||||||
|
Thanks Scott- So given an update like: db.collection.update( {shard_key: 1, _id: 1}, {_id: 1, shard_key: 1}) Will that populate shard_key on an insert? (nearly all of our writes are upserts, because we generate keys remotely) And for compound keys, I assume you'd need all of the values that are part of the key to be removed? Kiril Savino | |||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 09/May/13 ] | |||||||||||||||||||||
|
Kiril, if you remove the shard key from the update part, and just leave it in the query part then it should work. If you have a new case which is not working, please let us know. | |||||||||||||||||||||
| Comment by Kiril Savino [ 09/May/13 ] | |||||||||||||||||||||
|
Just to chime in, this is a serious breaking change. We just had to roll back our staging environment to 2.2, and will now be delaying our 2.4 rollout to product indefinitely, which really sucks. | |||||||||||||||||||||
| Comment by Pierre-Alban DEWITTE [ 06/May/13 ] | |||||||||||||||||||||
|
Thanks for your quick reply. | |||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 25/Apr/13 ] | |||||||||||||||||||||
|
We have been tightening up the restricts around a few corner case bugs where it was possible to accidentally change the shard key during the insert (if the document wasn't found and upsert was specified). Please remove the shard key from the update statement for now until we can diagnose and fix this on the server.
| |||||||||||||||||||||
| Comment by Pierre-Alban DEWITTE [ 25/Apr/13 ] | |||||||||||||||||||||
|
Hi, We face the same issue with a fresh migration from 2.2 to 2.4. Best regards |