[SERVER-6118] aggregation returns incorrect results for sharded sorts Created: 18/Jun/12  Updated: 11/Jul/16  Resolved: 16/Jul/12

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 2.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Mathias Stearn
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

The aggregation framework appears to distribute sorts to each shard and return sorted results from each shard sequentially, without ensuring that the combined result set is sorted.

Group, in contrast to sort, seems to be handled specifically within mongos.

Test:

// Set up a sharding test.
s = new ShardingTest( "aggregation_sort1", 2, 0, 2 );
s.adminCommand( { enablesharding:"test" } );
s.adminCommand( { shardcollection:"test.data", key:{ _id:1 } } );
 
db = s.getDB( "test" );
 
// Insert _id values 0 - 99.
N = 100;
for( i = 0; i < N; ++i ) {
    db.data.insert( { _id:i } )
}
db.getLastError();
 
// Split the data into 3 chunks.
s.adminCommand( { split:"test.data", middle:{ _id:33 } } );
s.adminCommand( { split:"test.data", middle:{ _id:66 } } );
 
// Migrate the middle chunk to another shard.
s.adminCommand( { movechunk:"test.data", find:{ _id:50 },
                to:s.getOther( s.getServer( "test" ) ).name } );
 
// Check that the results are in order.
result = db.data.aggregate( { $sort: { _id:1 } } ).result;
for( i = 0; i < N; ++i ) {
    assert.eq( i, result[ i ]._id );
}

Observed behavior: The results of a sharded sort aggregation may not be properly sorted.
Expected behavior: A merge sort is performed by mongos to sort the results.



 Comments   
Comment by auto [ 16/Jul/12 ]

Author:

{u'date': u'2012-07-16T14:21:01-07:00', u'email': u'mathias@10gen.com', u'name': u'Mathias Stearn'}

Message: Make test for SERVER-6118 more reliable by turning off balancer
Branch: master
https://github.com/mongodb/mongo/commit/f3ae34261a358a4ae4481c5d3ecf02d647c89fe4

Comment by auto [ 10/Jul/12 ]

Author:

{u'date': u'2012-07-06T17:56:22-07:00', u'email': u'mathias@10gen.com', u'name': u'Mathias Stearn'}

Message: SERVER-6118 support $sort in sharded aggregation
Branch: master
https://github.com/mongodb/mongo/commit/1dd2ddb7ec093843583de6343175885a5759ab79

Comment by auto [ 10/Jul/12 ]

Author:

{u'date': u'2012-07-06T17:09:58-07:00', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Make Pipeline::splitForSharded not O(n^2). Prep for SERVER-6118
Branch: master
https://github.com/mongodb/mongo/commit/6e7acaecc93697a9d8052db6814e17e0bf168841

Generated at Thu Feb 08 03:10:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.