[SERVER-18226] shardCollection command can orphan documents that are missing shard key if sparse index present Created: 27/Apr/15  Updated: 02/Sep/19  Resolved: 02/Sep/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: J Rassi Assignee: Kevin Pulo
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-10456 get cursor logic used to find docs to... Closed
Related
is related to SERVER-10456 get cursor logic used to find docs to... Closed
Operating System: ALL
Sprint: Sharding 2018-07-02, Sharding 2018-07-16, Sharding 2018-07-30, Sharding 2018-08-13, Sharding 2019-02-25, Sharding 2019-03-11, Sharding 2019-03-25, Sharding 2019-09-09
Participants:

 Description   

The shardCollection command verifies that all documents in the collection to be sharded contain some value of the shard key by delegating to the checkShardingIndex command. The checkShardingIndex implements this check by performing a full index scan on the first non-multikey index it finds prefixed with the shard key. However, if this index is a sparse index, this check will incorrectly pass if there are documents in the collection with a missing value for the shard key. These documents will subsequently be orphaned after the shardCollection command succeeds.

Reproduce with the following script, which fails when the "shardCollection" command incorrectly reports success. I tested that the script fails with 2.4.12/2.6.9/3.0.2/master. Note that the test passes when the creation order of the two indexes is swapped (this causes the correct index to be chosen for the task).

var st = new ShardingTest({shards: 1});
var coll = st.getDB("test").foo;
assert.writeOK(coll.insert({}));
assert.commandWorked(coll.ensureIndex({a: 1, b: 1}, {sparse: true}));
assert.commandWorked(coll.ensureIndex({a: 1}));
assert.commandWorked(coll.getDB().adminCommand({enableSharding: coll.getDB().getName()}));
assert.commandFailed(coll.getDB().adminCommand({shardCollection: coll.getFullName(), key: {a: 1}}));
assert.eq(1, coll.find().itcount());



 Comments   
Comment by Kevin Pulo [ 02/Sep/19 ]

Closing as a duplicate of SERVER-10456.

Generated at Thu Feb 08 03:47:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.