[SERVER-20768] Mongos find query including upper bound X of a chunk also targets the shard with chunk having lower bound = X Created: 05/Oct/15  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: 2.2.6, 3.0.6, 3.1.8
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-5365 range query on shard key incorrectly ... Closed
is duplicated by SERVER-38971 Single shard query hits multiple shar... Closed
is duplicated by SERVER-51731 ChunkManager::getShardIdsForRange sho... Closed
is duplicated by SERVER-4791 shard selection code ignores bound in... Closed
Assigned Teams:
Query Optimization
Operating System: ALL
Steps To Reproduce:

Start a ShardingTest as:

// sharding test MUST be started with noAutoSplit
var st = new ShardingTest({ mongos : 2, shards : 3, rs : { nodes : 3 }, other : { mongosOptions : { noAutoSplit: "" } } })

Then connect to a mongos in the cluster and run the following:

var db = db.getSiblingDB("test");
 
// insert some data
for (var i=0; i<1000; i++) {
    db.foo.insert({ _id : i });
}
 
// enable sharding
db.adminCommand({ enableSharding: db.getName() });
db.adminCommand({ shardCollection: "test.foo", key: { _id: 1 } });
 
// create two chunks
db.adminCommand({ split : "test.foo", middle: { _id : 1000 } });
 
// put the two chunks on separate shards
db.adminCommand({ moveChunk : "test.foo", find : { _id : 0 }, to : "test-rs0" });
db.adminCommand({ moveChunk : "test.foo", find : { _id : 1001 }, to : "test-rs1" });
 
// shows that both test-rs0 and test-rs1 are targeted
// even though the chunks are [-inf, 1000) and [1000, +inf)
printjson(db.foo.find({ _id : { $lt : 1000 } }).explain());

Participants:

 Description   

Chunks are inclusive at the lower bound and exclusive at the upper bound.

However, find queries over a range of the form

{ $lt : X }

where X is the upper bound of a chunk also targets the shard containing the chunk whose lower bound is X (at least according to find().explain()).

Note that a point query for X will only (and correctly) target the shard with the chunk whose lower bound is X.
Similarly, a query of the form

{ $lte : X }

will (correctly) target the shard for both chunks.

This is undesirable both from a performance perspective, since an additional shard is unnecessarily targeted in this situation, and a testing perspective, since .explain() cannot be used to verify that all documents within a chunk's range lie only on the shard the chunk is expected to be on.



 Comments   
Comment by James Hartig [ 14/Jan/19 ]

Given that this has a specific test for it, does that mean there's a reason it does this and there's no intention to ever fix it to correctly only hit one shard?

 

Additionally, this only appears to affect the $lt operator. If you do a $gt query with the same scenario, it correctly only targets a single shard.

Comment by David Storch [ 07/Dec/15 ]

I'll close the other one as the dup, since this report has more detail.

Comment by Randolph Tan [ 07/Dec/15 ]

david.storch yeah. It looks like a dup.

Comment by David Storch [ 07/Dec/15 ]

renctan, is this ticket a duplicate of SERVER-4791?

Comment by Randolph Tan [ 30/Oct/15 ]

It looks like this was a deliberate design decision since there is even a test for this exact case:

https://github.com/mongodb/mongo/blob/r3.1.9/src/mongo/s/chunk_manager_targeter_test.cpp#L422-l435

Comment by Randolph Tan [ 30/Oct/15 ]

This appears to be caused by ShardKeyPattern::flattenBounds:

https://github.com/mongodb/mongo/blob/r3.1.9/src/mongo/s/chunk_manager.cpp#L509

I have confirmed that in this case, the IndexBounds had the correct information of whether the bound is inclusive/exclusive. And the information is lost in flattenBounds.

Comment by Spencer Brody (Inactive) [ 05/Oct/15 ]

Assigning to Andy for triage

Generated at Thu Feb 08 03:55:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.