[SERVER-17426] Aggregation framework query by _id returns duplicates in sharded cluster (orphan documents) Created: 02/Mar/15  Updated: 24/Mar/15  Resolved: 11/Mar/15

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Querying
Affects Version/s: 2.6.8
Fix Version/s: 2.6.9

Type: Bug Priority: Major - P3
Reporter: Anil Kumar Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

mlaunch --sharded 2 --config 1 --single --binarypath <bin_version>
 
use duptest
db.dup.insert({_id: 2, a: 2, b: 2})
db.dup.insert({_id: 1, a: 1, b: 1})
db.dup.ensureIndex({a: 1})
sh.enableSharding("duptest")
sh.shardCollection("duptest.dup", {a: 1})
 
sh.splitAt("duptest.dup", {a: 2})
 
# If needed, perform moveChunk.
sh.moveChunk("duptest.dup", {a: 2}, "shard02")
 
 
# After split the MaxKey collection should be having the a = 2 document, on that shard add the document to have effect of orphan document
db.dup.insert({_id: 1, a: 1, b: 12})
 
# This results in two records while ideally it should be only 1
db.dup.aggregate([{$match: {_id: 1} }])
 
# This is expected to return 2 by its current design
db.dup.count({_id: 1})

Participants:

 Description   

In sharded cluster aggregation by _id alone results in duplicate records in case of orphan records exists on the other shards.

If the record with _id = 1 exists on the shard A (active) and shard B (orphan), the following aggregate will return 2 records.

db.test.aggregate([{$match: {_id: 1} }])

It just happens that this is the suggested way of getting an accurate (along with $group stage) count in sharded cluster in the documentation.

The incorrect behaviour is not noticed in 2.4.x and 3.0.0-rc11.



 Comments   
Comment by Githook User [ 11/Mar/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-17426 fix filtering of orphaned documents in IDHackRunner
Branch: v2.6
https://github.com/mongodb/mongo/commit/6b06804ca2a2d0bb292eb58510eb0f2808e879a4

Comment by Randolph Tan [ 09/Mar/15 ]

Problem with _idHack is the decision of whether to filter documents is done inside getNext() when iterating over a cursor. This is problematic because getMores are currently unversioned. Problem doesn't exist in v3.0 because the decision of whether to filter is done in planning stage.

Generated at Thu Feb 08 03:44:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.