[SERVER-8146] sort by array element is incorrect under sharding environment Created: 11/Jan/13  Updated: 04/Sep/15  Resolved: 04/Sep/15

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: 2.2.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: mongodbtudou Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: query_triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

centos 6.3


Issue Links:
Duplicate
Related
is related to SERVER-11878 Results in incorrect order for sharde... Closed
Operating System: ALL
Participants:

 Description   

the document is like the following:

{A:1, B:[{C:1, D:1}]}
{A:1, B:[{C:5, D:2}]}

my sort command is like:

db.xxx.find().sort({B.C:1})

it works well under single server mode, but the result may not be properly sorted under shard environment.

it seems the mongos only fetchs data from one sharded node serially, for exmpale:
shard1:

{A:1, B:[{C:4, D:1}]}
{A:1, B:[{C:5, D:2}]}

shard2:

{A:1, B:[{C:1, D:1}]}
{A:1, B:[{C:2, D:2}]}

after db.xxx.find().sort({B.C:1}), it returns like:

{A:1, B:[{C:4, D:1}]}
{A:1, B:[{C:5, D:2}]}
{A:1, B:[{C:1, D:1}]}
{A:1, B:[{C:2, D:2}]}

I'm not sure if it's the similar bug of https://jira.mongodb.org/browse/SERVER-6118, since it's on array element while 6118 is on non-array element.



 Comments   
Comment by David Storch [ 04/Sep/15 ]

The mongos query path is being rewritten under SERVER-15176. This includes an overhauled implementation of mongos sorting logic. We expect this to resolve issues related to delivering sorted .find() operations to a sharded cluster, so I am closing this as a duplicate of SERVER-15176.

Comment by mongodbtudou [ 15/Jan/13 ]

thanks for confirming!
The array sort feature is quite important to our new designed software.
Can someone point out which version is the begin of this regression? We may use the relative older one for our test temporarily,thanks!

Comment by Mathias Stearn [ 14/Jan/13 ]

I can confirm that this is an issue. It is not related to SERVER-6118 since that ticket is for the aggregation framework which handles this case correctly.

Here is some code to do a repro:

bigStr = 'x'; // to trigger splits
while (bigStr.length < 10*1024) { bigStr += bigStr; }
 
// db.bar sharded on {key:1}
for (var i =0; i<1000; i++) db.bar.insert({key:Math.random(), bigStr:bigStr, B: [{C:i}]})
 
// Normal query
mongos> db.bar.find({}, {bigStr:0}).sort( {'B.C': 1})
{ "_id" : ObjectId("50f474b5050b6651add24a2d"), "key" : 0.6599198130425066, "B" : [ { "C" : 1 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a34"), "key" : 0.7833042407874018, "B" : [ { "C" : 8 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a35"), "key" : 0.8530459373723716, "B" : [ { "C" : 9 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a37"), "key" : 0.8811661428771913, "B" : [ { "C" : 11 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a39"), "key" : 0.95997896627523, "B" : [ { "C" : 13 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3c"), "key" : 0.9075176373589784, "B" : [ { "C" : 16 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3f"), "key" : 0.6598584635648876, "B" : [ { "C" : 19 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a42"), "key" : 0.8530810354277492, "B" : [ { "C" : 22 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a44"), "key" : 0.9160628488752991, "B" : [ { "C" : 24 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a48"), "key" : 0.9078432768583298, "B" : [ { "C" : 28 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a49"), "key" : 0.6743440446443856, "B" : [ { "C" : 29 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a4b"), "key" : 0.6802574321627617, "B" : [ { "C" : 31 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a4c"), "key" : 0.698144412599504, "B" : [ { "C" : 32 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a4f"), "key" : 0.633671635761857, "B" : [ { "C" : 35 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a50"), "key" : 0.945851260330528, "B" : [ { "C" : 36 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a53"), "key" : 0.7673643978778273, "B" : [ { "C" : 39 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a55"), "key" : 0.5911061577498913, "B" : [ { "C" : 41 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a56"), "key" : 0.9130997934844345, "B" : [ { "C" : 42 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a58"), "key" : 0.8194703562185168, "B" : [ { "C" : 44 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a5b"), "key" : 0.6900255542714149, "B" : [ { "C" : 47 } ] }
Type "it" for more
 
// Sorting by whole-array 'B' rather than field 'B.C' works correctly:
mongos> db.bar.find({}, {bigStr:0}).sort( {B: 1})
{ "_id" : ObjectId("50f474b5050b6651add24a2c"), "key" : 0.26145654381252825, "B" : [ { "C" : 0 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a2d"), "key" : 0.6599198130425066, "B" : [ { "C" : 1 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a2e"), "key" : 0.3801005235873163, "B" : [ { "C" : 2 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a2f"), "key" : 0.29674924770370126, "B" : [ { "C" : 3 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a30"), "key" : 0.143473653588444, "B" : [ { "C" : 4 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a31"), "key" : 0.09666892886161804, "B" : [ { "C" : 5 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a32"), "key" : 0.2497417237609625, "B" : [ { "C" : 6 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a33"), "key" : 0.5610912672709674, "B" : [ { "C" : 7 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a34"), "key" : 0.7833042407874018, "B" : [ { "C" : 8 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a35"), "key" : 0.8530459373723716, "B" : [ { "C" : 9 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a36"), "key" : 0.5582540733739734, "B" : [ { "C" : 10 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a37"), "key" : 0.8811661428771913, "B" : [ { "C" : 11 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a38"), "key" : 0.2284426186233759, "B" : [ { "C" : 12 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a39"), "key" : 0.95997896627523, "B" : [ { "C" : 13 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3a"), "key" : 0.05256860516965389, "B" : [ { "C" : 14 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3b"), "key" : 0.18847709358669817, "B" : [ { "C" : 15 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3c"), "key" : 0.9075176373589784, "B" : [ { "C" : 16 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3d"), "key" : 0.3753254972398281, "B" : [ { "C" : 17 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3e"), "key" : 0.5363222123123705, "B" : [ { "C" : 18 } ] }
{ "_id" : ObjectId("50f474b5050b6651add24a3f"), "key" : 0.6598584635648876, "B" : [ { "C" : 19 } ] }
Type "it" for more
 
 
// Agg handles this case correctly:
 
db.bar.aggregate({$project: {key:1, 'B':1}}, {$sort: {'B.C': 1}}, {$limit:20}).result.forEach(printjsononeline)
{  "_id" : ObjectId("50f474b5050b6651add24a2c"),  "key" : 0.26145654381252825,  "B" : [         {       "C" : 0 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a2d"),  "key" : 0.6599198130425066,  "B" : [  {       "C" : 1 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a2e"),  "key" : 0.3801005235873163,  "B" : [  {       "C" : 2 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a2f"),  "key" : 0.29674924770370126,  "B" : [         {       "C" : 3 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a30"),  "key" : 0.143473653588444,  "B" : [   {       "C" : 4 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a31"),  "key" : 0.09666892886161804,  "B" : [         {       "C" : 5 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a32"),  "key" : 0.2497417237609625,  "B" : [  {       "C" : 6 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a33"),  "key" : 0.5610912672709674,  "B" : [  {       "C" : 7 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a34"),  "key" : 0.7833042407874018,  "B" : [  {       "C" : 8 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a35"),  "key" : 0.8530459373723716,  "B" : [  {       "C" : 9 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a36"),  "key" : 0.5582540733739734,  "B" : [  {       "C" : 10 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a37"),  "key" : 0.8811661428771913,  "B" : [  {       "C" : 11 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a38"),  "key" : 0.2284426186233759,  "B" : [  {       "C" : 12 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a39"),  "key" : 0.95997896627523,  "B" : [    {       "C" : 13 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3a"),  "key" : 0.05256860516965389,  "B" : [         {       "C" : 14 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3b"),  "key" : 0.18847709358669817,  "B" : [         {       "C" : 15 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3c"),  "key" : 0.9075176373589784,  "B" : [  {       "C" : 16 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3d"),  "key" : 0.3753254972398281,  "B" : [  {       "C" : 17 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3e"),  "key" : 0.5363222123123705,  "B" : [  {       "C" : 18 } ] }
{  "_id" : ObjectId("50f474b5050b6651add24a3f"),  "key" : 0.6598584635648876,  "B" : [  {       "C" : 19 } ] }
 
 

Generated at Thu Feb 08 03:16:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.