Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18226

shardCollection command can orphan documents that are missing shard key if sparse index present

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Sharding
    • Labels:
      None
    • Operating System:
      ALL
    • Sprint:
      Sharding 2018-07-02, Sharding 2018-07-16, Sharding 2018-07-30, Sharding 2018-08-13, Sharding 2019-02-25, Sharding 2019-03-11, Sharding 2019-03-25, Sharding 2019-09-09

      Description

      The shardCollection command verifies that all documents in the collection to be sharded contain some value of the shard key by delegating to the checkShardingIndex command. The checkShardingIndex implements this check by performing a full index scan on the first non-multikey index it finds prefixed with the shard key. However, if this index is a sparse index, this check will incorrectly pass if there are documents in the collection with a missing value for the shard key. These documents will subsequently be orphaned after the shardCollection command succeeds.

      Reproduce with the following script, which fails when the "shardCollection" command incorrectly reports success. I tested that the script fails with 2.4.12/2.6.9/3.0.2/master. Note that the test passes when the creation order of the two indexes is swapped (this causes the correct index to be chosen for the task).

      var st = new ShardingTest({shards: 1});
      var coll = st.getDB("test").foo;
      assert.writeOK(coll.insert({}));
      assert.commandWorked(coll.ensureIndex({a: 1, b: 1}, {sparse: true}));
      assert.commandWorked(coll.ensureIndex({a: 1}));
      assert.commandWorked(coll.getDB().adminCommand({enableSharding: coll.getDB().getName()}));
      assert.commandFailed(coll.getDB().adminCommand({shardCollection: coll.getFullName(), key: {a: 1}}));
      assert.eq(1, coll.find().itcount());
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              kevin.pulo Kevin Pulo
              Reporter:
              rassi J Rassi
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: