[JAVA-2217] MongoCursor - maxTime - TimeoutException is thrown after wrong period of time Created: 10/Jun/16  Updated: 27/Feb/17  Resolved: 23/Sep/16

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: 3.2.2
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Krzysztof Barczyk Assignee: Ross Lawley
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Please have a look at following code snipped:

MongoDatabase db = mongoClient.getDatabase("local");
MongoCollection<Document> oplogCollection = db.getCollection("oplog.$main");
 
MongoCursor<Document> cursor = oplogCollection.find(aFilter())
                .cursorType(CursorType.Tailable)
                //will throw MongoExecutionTimeoutException after ~ 10s -30s
                .maxTime(300, MILLISECONDS)
                .iterator();
 
//following code will produce an exception after like 10-30s:
cursor.hasNext(); 

I tried to implement this functionality using cursor type=TailableAwait and maxAwaitTile(300, MILLISECONDS) but in such case code was just freezing without any timeout.



 Comments   
Comment by Mouthgalya Ganapathy [ 27/Feb/17 ]

Thanks a lot Ross. I have posted it in stackoverflow too. It would be great if you could spend few minutes and answer that.
http://stackoverflow.com/questions/42471841/mongo-oplog-tailable-cursor
I will post it in mongodb user mailing list too.

Thanks in advance.,
Mouthgalya Ganapathy

Comment by Ross Lawley [ 27/Feb/17 ]

Hi mganapat,

This is an old ticket that has been closed. Please ask your questions on the mongodb-user mailinglist or on stackoverflow and we'll endeavour to follow up there.

All the best,

Ross

Comment by Mouthgalya Ganapathy [ 26/Feb/17 ]

As a follow up clarification for the same exception below
com.mongodb.MongoExecutionTimeoutException: operation exceeded time limit

a.) Would setting the maxTime help in handling the exception better?? What would be a practical value for maxTime given that we are using a tailable await cursor? The below link is indicating that for a cursor subsequent "getmore" requests will be included in the total time. https://www.mongodb.com/blog/post/maxtimems-and-query-optimizer-introspection-in

b.) How would using a non blocking cursor call help for tailable await cursor?? Would it help in bypassing the waiting? http://mongodb.github.io/mongo-java-driver/3.4/javadoc/com/mongodb/client/MongoCursor.html#tryNext--

c.) In case of an exception like above, what is the best way to handle the error gracefully and continue the processing of subsequent records?

Comment by Ross Lawley [ 23/Sep/16 ]

Hi krzysztofbarczyk,

Many apologies for taking so long to look into this issue in detail.

The maxTime setting specifies a cumulative time limit for processing operations on a cursor. As noted in the cursor.maxTimeMS documentation neither cursor idle time or network latency contribute towards its processing time.

This accounts for the observation of seeing the cursor take longer to throw a MongoExecutionTimeoutException.

TailableAwait cursors are also a special case as each getMore call to the server will set a maxTime, thus increasing the amount of observable time for an operation to complete. By design it will await in a busy loop for more data.

The MongoCursor#tryNext method is especially useful for tailable cursors as it will immediately return with either a document or a null.

Because of how the server has implemented maxTime, I'm marking this ticket as works as designed for the reasons mentioned above.

Comment by Krzysztof Barczyk [ 20/Jul/16 ]

Hi, thanks for your response.

version of Java driver is: 3.2.2,
version of MongoDB is 3.2.0

Comment by Ross Lawley [ 07/Jul/16 ]

Apologies for the delay krzysztofbarczyk, could you confirm the version of MongoDB you are running this code against?

Comment by Krzysztof Barczyk [ 07/Jul/16 ]

Hi, is there any news? Were you able to reproduce the issue?

Generated at Thu Feb 08 08:56:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.