[SERVER-22849] Shard registry should update config last visible opTime even on command errors Created: 25/Feb/16  Updated: 19/Nov/16  Resolved: 27/Feb/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.3.2
Fix Version/s: 3.2.4, 3.3.3

Type: Bug Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Kaloian Manassiev
Resolution: Done Votes: 0
Labels: code-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to SERVER-22590 applyChunkOpsDeprecated retries and t... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Sprint: Sharding 11 (03/11/16)
Participants:
Linked BF Score: 0

 Description   

The CSRS catalog manager applyChunkOpsDeprecated call will retry operations, which fail because of transient errors. If the retry fails because of mismatched condition, we will try to check if the correct chunk version is present. This check may go against a secondary and because we did not update the last visible opTime won't see the correct value. Hence we get this error:

[js_test:split_with_force] 2016-02-25T21:00:25.031+0000 d20260| 2016-02-25T21:00:25.015+0000 W SHARDING [conn5] chunk operation commit failed and metadata will be revalidated :: caused by :: BadValue: pre-condition failed
...
[js_test:split_with_force] 2016-02-25T21:00:26.960+0000 s20264| 2016-02-25T21:00:26.912+0000 W SHARDING [conn1] splitChunk cmd { splitChunk: "foo.bar", keyPattern: { _id: 1.0 }, min: { _id: MinKey }, max: { _id: MaxKey }, from: "shard0000", splitKeys: [ { _id: 0.0 } ], configdb: "test-configRS/ip-10-153-175-231:20261,ip-10-153-175-231:20262,ip-10-153-175-231:20263", shardVersion: [ Timestamp 1000|0, ObjectId('56cf6b620db8dce39f52eb7f') ], epoch: ObjectId('56cf6b620db8dce39f52eb7f') } failed :: caused by :: BadValue: chunk operation commit failed: version 1|2||56cf6b620db8dce39f52eb7f doesn't exist in namespacefoo.bar. Unable to save chunk ops. Command: { applyOps: [ { op: "u", b: true, ns: "config.chunks", o: { _id: "foo.bar-_id_MinKey", lastmod: Timestamp 1000|1, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: MinKey }, max: { _id: 0.0 }, shard: "shard0000" }, o2: { _id: "foo.bar-_id_MinKey" } }, { op: "u", b: true, ns: "config.chunks", o: { _id: "foo.bar-_id_0.0", lastmod: Timestamp 1000|2, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: 0.0 }, max: { _id: MaxKey }, shard: "shard0000" }, o2: { _id: "foo.bar-_id_0.0" } } ], preCondition: [ { ns: "config.chunks", q: { query: { ns: "foo.bar" }, orderby: { lastmod: -1 } }, res: { lastmod: Timestamp 1000|0 } } ], writeConcern: { w: "majority", wtimeout: 15000 } }. Result: { got: { _id: "foo.bar-_id_0.0", lastmod: Timestamp 1000|2, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: 0.0 }, max: { _id: MaxKey }, shard: "shard0000" }, whatFailed: { ns: "config.chunks", q: { query: { ns: "foo.bar" }, orderby: { lastmod: -1 } }, res: { lastmod: Timestamp 1000|0 } }, ok: 0.0, errmsg: "pre-condition failed", code: 2 }



 Comments   
Comment by Githook User [ 29/Feb/16 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-22849 Always update the visible opTime

This change makes sure that the ShardRegistry does not skip updating the
config server visible opTime, even if the commands come back with
command-specific errors.

(cherry picked from commit 81efb319f05f4ec1b5bd4523b13694241e967fc2)
Branch: v3.2
https://github.com/mongodb/mongo/commit/a263d39b03a1cf4f199745e0afe06a0d8c734518

Comment by Githook User [ 29/Feb/16 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-22849 Improve logging for splitChunk

(cherry picked from commit e69637fe7037718b43100a9e3c2d053f9b6520ce)
Branch: v3.2
https://github.com/mongodb/mongo/commit/5866f2bd0c8a76ea61d343e25a876efa39a21d41

Comment by Githook User [ 27/Feb/16 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-22849 Always update the visible opTime

This change makes sure that the ShardRegistry does not skip updating the
config server visible opTime, even if the commands come back with
command-specific errors.
Branch: master
https://github.com/mongodb/mongo/commit/81efb319f05f4ec1b5bd4523b13694241e967fc2

Comment by Githook User [ 26/Feb/16 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-22849 Improve logging for splitChunk
Branch: master
https://github.com/mongodb/mongo/commit/e69637fe7037718b43100a9e3c2d053f9b6520ce

Generated at Thu Feb 08 04:01:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.