Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22849

Shard registry should update config last visible opTime even on command errors

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.3.2
    • Fix Version/s: 3.2.4, 3.3.3
    • Component/s: Sharding
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Sprint:
      Sharding 11 (03/11/16)
    • Linked BF Score:
      0

      Description

      The CSRS catalog manager applyChunkOpsDeprecated call will retry operations, which fail because of transient errors. If the retry fails because of mismatched condition, we will try to check if the correct chunk version is present. This check may go against a secondary and because we did not update the last visible opTime won't see the correct value. Hence we get this error:

      [js_test:split_with_force] 2016-02-25T21:00:25.031+0000 d20260| 2016-02-25T21:00:25.015+0000 W SHARDING [conn5] chunk operation commit failed and metadata will be revalidated :: caused by :: BadValue: pre-condition failed
      ...
      [js_test:split_with_force] 2016-02-25T21:00:26.960+0000 s20264| 2016-02-25T21:00:26.912+0000 W SHARDING [conn1] splitChunk cmd { splitChunk: "foo.bar", keyPattern: { _id: 1.0 }, min: { _id: MinKey }, max: { _id: MaxKey }, from: "shard0000", splitKeys: [ { _id: 0.0 } ], configdb: "test-configRS/ip-10-153-175-231:20261,ip-10-153-175-231:20262,ip-10-153-175-231:20263", shardVersion: [ Timestamp 1000|0, ObjectId('56cf6b620db8dce39f52eb7f') ], epoch: ObjectId('56cf6b620db8dce39f52eb7f') } failed :: caused by :: BadValue: chunk operation commit failed: version 1|2||56cf6b620db8dce39f52eb7f doesn't exist in namespacefoo.bar. Unable to save chunk ops. Command: { applyOps: [ { op: "u", b: true, ns: "config.chunks", o: { _id: "foo.bar-_id_MinKey", lastmod: Timestamp 1000|1, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: MinKey }, max: { _id: 0.0 }, shard: "shard0000" }, o2: { _id: "foo.bar-_id_MinKey" } }, { op: "u", b: true, ns: "config.chunks", o: { _id: "foo.bar-_id_0.0", lastmod: Timestamp 1000|2, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: 0.0 }, max: { _id: MaxKey }, shard: "shard0000" }, o2: { _id: "foo.bar-_id_0.0" } } ], preCondition: [ { ns: "config.chunks", q: { query: { ns: "foo.bar" }, orderby: { lastmod: -1 } }, res: { lastmod: Timestamp 1000|0 } } ], writeConcern: { w: "majority", wtimeout: 15000 } }. Result: { got: { _id: "foo.bar-_id_0.0", lastmod: Timestamp 1000|2, lastmodEpoch: ObjectId('56cf6b620db8dce39f52eb7f'), ns: "foo.bar", min: { _id: 0.0 }, max: { _id: MaxKey }, shard: "shard0000" }, whatFailed: { ns: "config.chunks", q: { query: { ns: "foo.bar" }, orderby: { lastmod: -1 } }, res: { lastmod: Timestamp 1000|0 } }, ok: 0.0, errmsg: "pre-condition failed", code: 2 }
      

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: