[JAVA-1551] DBCursor slowness Created: 14/Nov/14  Updated: 11/Sep/19  Resolved: 14/Nov/14

Status: Closed
Project: Java Driver
Component/s: Performance
Affects Version/s: 2.12.3
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Mike Gozaloff Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

RedHat server windows client



 Description   

We are using Spring Data with the MongoDB Java driver.
We create an aggregation using AggregationOptions.OutputMode.CURSOR
There are 2 fields in the projection on a collection with almost 800K documents.
The aggregation does not use a sort or group; only a match and projection.
The aggregation matches on a single String value that is indexed.
The expected result set is all of the documents except 2 (over 700K).
The DBCursor.next() method seems very slow. I've tried a smaller data set with OutputMode.INLINE and the next method is much faster.

Here is the entire loop:
while (cursor.hasNext()) {
DBObject obj = cursor.next();
logger.warn("count=" + count++);
if (count % 10000 == 0)

{ logger.warn("count=" + count); }

}

Here are some timing from the logs:

14 Nov 2014 14:44:56,328 WARN : TestDistributionFilter,207 - count=6133
14 Nov 2014 14:44:56,418 WARN : TestDistributionFilter,207 - count=6134
14 Nov 2014 14:44:56,508 WARN : TestDistributionFilter,207 - count=6135
14 Nov 2014 14:44:56,597 WARN : TestDistributionFilter,207 - count=6136
14 Nov 2014 14:44:56,686 WARN : TestDistributionFilter,207 - count=6137
14 Nov 2014 14:44:56,776 WARN : TestDistributionFilter,207 - count=6138

It seems odd that it is taking almost 100 milisecs to pull back a single record from the server. I've commented out the record logging but that did not help much.

What can we do to speed it up?

Thanks,

gozer



 Comments   
Comment by Jeffrey Yemin [ 14/Nov/14 ]

Glad that you figured it out.

Comment by Mike Gozaloff [ 14/Nov/14 ]

I was able in increase the speed by adjusting the batch size. The sample code I copied had it set to 1 which is less than optimal. You can go ahead and close this issue.

Thanks,

gozer

Comment by Mike Gozaloff [ 14/Nov/14 ]

I realize that google groups is the preferred starting point. The problem is that the bank blocks access to google groups. I've been digging through stackoverflow and other various posts. Have not found a similar posting.

Thanks,

gozer

Comment by Jeffrey Yemin [ 14/Nov/14 ]

Thanks for the report. Do you have reason to suspect that this is an issue with the Java driver itself? Otherwise, it's best to start off discussions like this in the mongodb-user forum. From there someone can help you to determine whether there is a problem with your application or whether a SERVER or JAVA issue should be opened.

Regards,
Jeff

Generated at Thu Feb 08 08:54:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.