[SERVER-4848] limit() is returning less than it should at sharded collection Created: 02/Feb/12  Updated: 11/Jul/16  Resolved: 10/Jan/13

Status: Closed
Project: Core Server
Component/s: Sharding, Shell
Affects Version/s: 1.8.3, 2.0.2
Fix Version/s: 2.0.8

Type: Bug Priority: Major - P3
Reporter: HAMADA Ran Assignee: Greg Studer
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS 5.7 (64bit)


Attachments: File limit_return.js     File test.groovy     File test.js     File test.pl     Text File test_groovy-result.txt     Text File test_pl-result.txt    
Operating System: ALL
Participants:

 Description   

I tried query to sharded collection with limit() method at mongo shell, and sometimes got less documents than it should.

Test code (attached below):

test.js

db = connect("localhost:27017/myTest");
 
print( "Prepareing test data" );
db.adminCommand( {enablesharding: "myTest"});
db.myColl.drop();
db.adminCommand( {shardcollection: "myTest.myColl", key: {key:1}} );
var padding = ''; for (var i=0; i<10000; i++){ padding += '*'; }
for (var i=0; i<5000; i++){ db.myColl.save({key: 0, value: i, padding: padding}); }
 
var conf = db.getSisterDB("config");
print( "Number of the shards : " + conf.shards.count() );
print( "Chunks : " );
conf.chunks.find({ns:"myTest.myColl"}).forEach(printjson);
print( "Test result:" );
 
for(var lm=100; lm<2000; lm+=100){
        var i=0;db.myColl.find().limit(lm).forEach(function(x){i=i+1});
        print(lm + ":" + i + (lm == i ? "" : " <- !?"))
}

and result :

$ mongo --nodb test.js
MongoDB shell version: 2.0.2
connecting to: localhost:27017/myTest
Prepareing test data
Number of the shards : 3
Chunks :
        "_id" : "myTest.myColl-key_MinKey",
        "lastmod" : {
                "t" : 1000,
                "i" : 0
        },
        "ns" : "myTest.myColl",
        "min" : {
                "key" : { $minKey : 1 }
        },
        "max" : {
                "key" : { $maxKey : 1 }
        },
        "shard" : "shard1" 
}
Test result:
100:100
200:115 <- !?
300:300
400:400
500:500
600:600
700:554 <- !?
800:454 <- !?
900:900
1000:1000
1100:1100
1200:993 <- !?
1300:893 <- !?
1400:793 <- !?
1500:1500
1600:1600
1700:1700
1800:1332 <- !?
1900:1232 <- !?

I tried same test via mondod(shard) directly and it's OK.
I tried same test in single not sharded collection and it's OK.

I tried same test with Perl driver (MongoDB 0.45 CPAN module) and it's OK. (test code and result attached)



 Comments   
Comment by Greg Studer [ 10/Jan/13 ]

Attached a slightly easier-to-run version of the script. Later versions of 2.0, 2.2, and 2.4 no longer manifest this error, so marking as fixed.

Comment by Greg Studer [ 28/Jun/12 ]

Unclear - there were some changes backported, more likely in later 2.0.x versions. However, hamada's test case (test.js) makes it very simple to run on your own mongo installation - we're still working on this.

Comment by Shirish Reddy P [ 28/Jun/12 ]

Does this affect 2.0.4 too ?

Comment by Greg Studer [ 12/Apr/12 ]

I can repro in 2.0.2 - not in 2.1.1 - this may be related to some getMore changes that got pushed a few weeks ago. Will follow up.

Comment by HAMADA Ran [ 02/Feb/12 ]

I had attached 2 files.
test.groovy
test_groovy-result.txt

There are the same test using Java driver 2.7.2, and has same problem as mongo shell (JavaScript) does.

Comment by HAMADA Ran [ 02/Feb/12 ]

Same test using Java Driver 2.7.2

Generated at Thu Feb 08 03:07:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.