[SERVER-6015] using sort() with batchSize() never returns an active cursor, therefore incomplete resultsets can be returned Created: 06/Jun/12  Updated: 12/Jun/14  Resolved: 06/Jun/12

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.0.5, 2.1.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Lee Henson Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

tried via mongoshell & java driver 2.6.5/2.8.0rc1 on osx lion (installed via homebrew) & ubuntu 10.10


Issue Links:
Duplicate
duplicates SERVER-5374 batchSize is a hard limit for an in m... Closed
Related
related to SERVER-14228 Setting batchSize and sort on a curso... Closed
Operating System: ALL
Participants:

 Description   

I can demonstrate the problem using mongoshell commands. First the case which proves the batchsize works when no sort criteria is provided:

Given a collection containing 101 documents
when I issue a query which matches all documents in the collection, using a batchSize which is smaller than number of documents in the collection
and I iterate through the cursor, keeping a count of the number of documents
then the count should match the number of documents in the collection.

 
> db.test.count()
101
 
> var cursor = db.test.find().batchSize(100);
> var count = 0;
> while(cursor.hasNext()) { cursor.next(); count++; }
100
 
> count
101

mongosniff output:

127.0.0.1:54977  -->> 127.0.0.1:27017 ruby-test-db.test  51 bytes  id:a	10
	query: {}  ntoreturn: 100 ntoskip: 0
127.0.0.1:27017  <<--  127.0.0.1:54977 ?e?*?|y  13869 bytes  id:2667	9831 - 10
	reply n:100 cursorId: 8754037985523367853
	{ _id: ObjectId('4fce22c3977225f8cb3b5ad1'), A1338909379446: 1338909379.447, A1338909379447: 1338909379.448, A1338909379448: 1338909379.449, A1338909379449: 1338909379.449, xyz: 1, abc: 330 }
127.0.0.1:54977  -->> 127.0.0.1:27017 ruby-test-db.test  50 bytes  id:b	11
	getMore nToReturn: 100 cursorId: 8754037985523367853
127.0.0.1:27017  <<--  127.0.0.1:54977   227 bytes  id:2668	9832 - 11
	reply n:1 cursorId: 0
	{ _id: ObjectId('4fce22c3977225f8cb3b5aea'), A1338909379578: 1338909379.584, A1338909379584: 1338909379.585, A1338909379585: 1338909379.588, A1338909379588: 1338909379.589, A1338909379589: 1338909379.59, A133890937959: 1338909379.59, xyz: 1, abc: 30 }

Now, when I supply a sort criteria to the query, the batchsize acts as a limit and immediately closes the cursor on the server.

> var cursor = db.test.find().sort({ abc: 1 }).batchSize(100);
 
> var count = 0;
> while(cursor.hasNext()) { cursor.next(); count++; }
99
 
> count
100

mongosniff output:

query: { query: {}, orderby: { abc: 1.0 } }  ntoreturn: 100 ntoskip: 0
127.0.0.1:27017  <<--  127.0.0.1:54977   13916 bytes  id:2678	9848 - 27
	reply n:100 cursorId: 0
	{ _id: ObjectId('4fce22c3977225f8cb3b5acf'), A1338909379437: 1338909379.438, A1338909379438: 1338909379.439, A1338909379439: 1338909379.44, A133890937944: 1338909379.44, xyz: 1, abc: 2 }

In this scenario, any further results past the end of the first returned batch are effectively ignored. Not good if you need them!



 Comments   
Comment by Lee Henson [ 06/Jun/12 ]

Thanks Aaron, adding an index has solved it!

Comment by Aaron Staple [ 06/Jun/12 ]

Hi Lee - If you don't have an index for your sort the current behavior is for batch size to be a hard limit. This is SERVER-5374. Please reopen if this is a different issue.

Generated at Thu Feb 08 03:10:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.