[JAVA-2970] How Mongo Java Driver decide to invoke GetMore? Created: 10/Sep/18  Updated: 11/Sep/19  Resolved: 10/Sep/18

Status: Closed
Project: Java Driver
Component/s: Connection Management
Affects Version/s: 3.5.0
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: RameshPitani Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi,

We are having a weird issue with Mongo java driver 3.5.0. We have following error initially returned by Mongo and application is handling error smoothly. But after this error, whatever next request picks mongo connection, Next request is hanging forever and not timing out after a certain interval.

 

# client-options
mongo.connectionsPerHost=20
mongo.threadsAllowedToBlockForConnectionMultiplier=2
mongo.connectTimeout=30000
mongo.maxWaitTime=10000
mongo.socketKeepAlive=true
mongo.socketTimeout=20000
mongo.maxConnectionIdleTime=300000
mongo.maxConnectionLifeTime=43200000
mongo.heartbeatFrequency=3600000
mongo.minHeartbeatFrequency=500
mongo.heartbeatConnectTimeout=20000
mongo.heartbeatSocketTimeout=20000

 

Caused by: com.mongodb.MongoQueryException: 
  Query failed with error code 35 and error message '' on server localhost:27017 at
com.mongodb.connection.ProtocolHelper.getQueryFailureException(ProtocolHelper.java:131) at
com.mongodb.connection.GetMoreProtocol.execute(GetMoreProtocol.java:88) at 
com.mongodb.connection.GetMoreProtocol.execute(GetMoreProtocol.java:49) at 
com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159) at 
com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289) at 
com.mongodb.connection.DefaultServerConnection.getMore(DefaultServerConnection.java:254) at 
com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:218) at 
com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:105) at
com.mongodb.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:46) at 
com.mongodb.DBCursor.hasNext(DBCursor.java:145) at 
com.mongodb.DBCursor.fillArray(DBCursor.java:910) at 
com.mongodb.DBCursor.toArray(DBCursor.java:655) at 
com.mongodb.DBCursor.toArray(DBCursor.java:643)

 



 Comments   
Comment by RameshPitani [ 15/Sep/18 ]

Thank you. So GetMore is invoked based on BatchSize set on DBCursor. If BatchSize is not set on DBSursor, then what is the default value of BatchSize?

Comment by Jeffrey Yemin [ 10/Sep/18 ]

The driver executes a getMore command when the previous batch has been iterated, e.g.

 

 
// hasNext will call getMore when all the documents from the previous batch have been iterated
 
while (cursor.hasNext()) {
 
  cursor.next();

Comment by RameshPitani [ 10/Sep/18 ]

Jeff,

Request type is the Question. We are aware there is an issue in Z/TPF Mongo server. 

I am trying to understand from mongo java Driver(3.5.0) perspective. What makes the driver invoke below method? Is Client decide or Server tells client application to make another request to get more data?

2018-08-26 12:32:58,891 [pool-1-thread-19] DEBUG org.mongodb.driver.protocol.getmore () Getting more documents from namespace tpfdf.LF31AA with cursor 1 on connection [connectionId

{localValue:14, serverValue:0}

] to server localhost:27017

Comment by Jeffrey Yemin [ 10/Sep/18 ]

I'm going to change the issue type to Question, since it appears that the Java driver itself is operating correctly by reporting an error returned by the server.  If it turns out this is a server issue, we can move it and change the type accordingly.

Comment by Jeffrey Yemin [ 10/Sep/18 ]

Hi pitani

Sorry to hear you're having trouble.  Can you let us know what version of the MongoDB server your application is connected to, and what type of cluster it is (standalone, replica set, sharded).

Error code 35 in the latest server version maps to UserModificationFailed so this is a somewhat surprising error to encounter for a normal query.

 

Comment by RameshPitani [ 10/Sep/18 ]

Some times we are seeing org.mongodb.driver.protocol.getmore () in log files, sometimes we are not seeing getMore() in the log files. What configuration trigger this getmore() invocation and how can we stop this getmore() invocation.

Comment by RameshPitani [ 10/Sep/18 ]

2018-08-26 12:32:56,652 [pool-1-thread-19] DEBUG TpfdfRepositoryImpl.class () find using query: \{ "_id" : { "$in" : [ { "$oid" : "0000000000000000fa853f41"} , \{ "$oid" : "0000000000000000fa86223f"} , \{ "$oid" : "0000000000000000fa85323c" } , \{ "$oid" : "0000000000000000fa854746"} , \{ "$oid" : "0000000000000000fa861a49"} , \{ "$oid" : "0000000000000000fa862043"} , \{ "$oid" : "0000000000000000fa861a51"} , \{ "$oid" : "0000000000000000fa853451"} , \{ "$oid" : "0000000000000000 fa853c50"} , \{ "$oid" : "0000000000000000fa856e4b"} , \{ "$oid" : "0000000000000000fa853c4b"} , \{ "$oid" : "0000000000000000fa861953"} , \{ "$oid" : "0000000000000000fa85325e"} , \{ "$oid" : "0000000000000000fa853260"} , \{ "$oid" : "0000000 000000000fa86195c"} , \{ "$oid" : "0000000000000000fa853267"} , \{ "$oid" : "0000000000000000fa853266"} , \{ "$oid" : "0000000000000000fa854763"} , \{ "$oid" : "0000000000000000fa853c6f"} , \{ "$oid" : "0000000000000000fa85476f"} , \{ "$oid" : "0000000000000000fa853c6e"} , \{ "$oid" : "0000000000000000fa853c70"} , \{ "$oid" : "0000000000000000fa853c6d"} , \{ "$oid" : "0000000000000000fa861a6a"} , \{ "$oid" : "0000000000000000fa85476c"} , \{ "$oid" : "0000000000000000fa853c6c"} , \{ "$oid" : "0000000000000000fa861a72"} , \{ "$oid" : "0000000000000000fa861a7f"} , \{ "$oid" : "0000000000000000fa854780"} , \{ "$oid" : "0000000000000000fa854787"} , \{ "$oid" : "0000000000000000fa854987"} , \{ "$oid" : "0000000000000000fa861 a89"} , \{ "$oid" : "0000000000000000fa861c84"} , \{ "$oid" : "0000000000000000fa854783"} , \{ "$oid" : "0000000000000000fa861a82"} , \{ "$oid" : "0000000000000000fa854784"} , \{ "$oid" : "0000000000000000fa861a83"} , \{ "$oid" : "000000000000 0000fa85478e"} , \{ "$oid" : "0000000000000000fa853c8a"} , \{ "$oid" : "0000000000000000fa853e8a"} , \{ "$oid" : "0000000000000000fa861a8b"} , \{ "$oid" : "0000000000000000fa861a98"} , \{ "$oid" : "0000000000000000fa861a99"} , \{ "$oid" : "000 0000000000000fa861a9f"} , \{ "$oid" : "0000000000000000fa861a9a"} , \{ "$oid" : "0000000000000000fa853ea6"} , \{ "$oid" : "0000000000000000fa8547a5"} , \{ "$oid" : "0000000000000000fa8621b0"} , \{ "$oid" : "0000000000000000fa8621ad"} , \{ "$oi d" : "0000000000000000fa8547b3"} , \{ "$oid" : "0000000000000000fa8547b4"} , \{ "$oid" : "0000000000000000fa8547bf"} , \{ "$oid" : "0000000000000000fa8621c1"} , \{ "$oid" : "0000000000000000fa860cbe"} , \{ "$oid" : "0000000000000000fa8621bf"} , \{ "$oid" : "0000000000000000fa8547bb"} , \{ "$oid" : "0000000000000000fa8547bd"} , \{ "$oid" : "0000000000000000fa860cbb"} , \{ "$oid" : "0000000000000000fa860cc8"} , \{ "$oid" : "0000000000000000fa8621c9"} , \{ "$oid" : "0000000000000000f a8621c6"} , \{ "$oid" : "0000000000000000fa860cc4"} , \{ "$oid" : "0000000000000000fa860cc2"} , \{ "$oid" : "0000000000000000fa853bd1"} , \{ "$oid" : "0000000000000000fa860ccd"} , \{ "$oid" : "0000000000000000fa8621cb"} , \{ "$oid" : "00000000 00000000fa860ccb"} , \{ "$oid" : "0000000000000000fa860cd8"} , \{ "$oid" : "0000000000000000fa8621d9"} , \{ "$oid" : "0000000000000000fa853bd6"} , \{ "$oid" : "0000000000000000fa860cd6"} , \{ "$oid" : "0000000000000000fa860cd7"} , \{ "$oid" : "0000000000000000fa860cd5"} , \{ "$oid" : "0000000000000000fa860cd2"} , \{ "$oid" : "0000000000000000fa860cd3"} , \{ "$oid" : "0000000000000000fa8621e1"} , \{ "$oid" : "0000000000000000fa8621df"} , \{ "$oid" : "0000000000000000fa8611dd"} , \{ "$oid" : "0000000000000000fa8611da"} , \{ "$oid" : "0000000000000000fa8611db"} , \{ "$oid" : "0000000000000000fa853bdc"} , \{ "$oid" : "0000000000000000fa853be6"} , \{ "$oid" : "0000000000000000fa8621e6"} , \{ "$oid" : "0000000000000000fa860c e7"} , \{ "$oid" : "0000000000000000fa853be2"} , \{ "$oid" : "0000000000000000fa8621e5"} , \{ "$oid" : "0000000000000000fa8644f0"} , \{ "$oid" : "0000000000000000fa8618ef"} , \{ "$oid" : "0000000000000000fa860cea"} , \{ "$oid" : "0000000000000 000fa8644ea"} , \{ "$oid" : "0000000000000000fa860cf8"} , \{ "$oid" : "0000000000000000fa856df6"} , \{ "$oid" : "0000000000000000fa8621f7"} , \{ "$oid" : "0000000000000000fa8619f7"} , \{ "$oid" : "0000000000000000fa860cf7"} , \{ "$oid" : "0000 000000000000fa856df8"} , \{ "$oid" : "0000000000000000fa856df5"} , \{ "$oid" : "0000000000000000fa8644f2"} , \{ "$oid" : "0000000000000000fa8621fa"} , \{ "$oid" : "0000000000000000fa8644fb"} , \{ "$oid" : "0000000000000000fa860d09"} , \{ "$oid " : "0000000000000000fa854806"} , \{ "$oid" : "0000000000000000fa854708"} , \{ "$oid" : "0000000000000000fa854705"} , \{ "$oid" : "0000000000000000fa860d0e"} , \{ "$oid" : "0000000000000000fa85350a"} , \{ "$oid" : "0000000000000000fa85480a"} , \{ "$oid" : "0000000000000000fa85480d"} , \{ "$oid" : "0000000000000000fa86450b"} , \{ "$oid" : "0000000000000000fa861918"} , \{ "$oid" : "0000000000000000fa861919"} , \{ "$oid" : "0000000000000000fa861916"} , \{ "$oid" : "0000000000000000fa 853c19"} , \{ "$oid" : "0000000000000000fa861917"} , \{ "$oid" : "0000000000000000fa85321e"} , \{ "$oid" : "0000000000000000fa853221"} , \{ "$oid" : "0000000000000000fa85321b"} , \{ "$oid" : "0000000000000000fa853c1b"} , \{ "$oid" : "000000000 0000000fa85471b"} , \{ "$oid" : "0000000000000000fa85321a"} , \{ "$oid" : "0000000000000000fa85471d"} , \{ "$oid" : "0000000000000000fa85471c"} , \{ "$oid" : "0000000000000000fa85321c"} , \{ "$oid" : "0000000000000000fa853230"} , \{ "$oid" : " 0000000000000000fa85472b"} , \{ "$oid" : "0000000000000000fa85322d"} , \{ "$oid" : "0000000000000000fa856e36"} , \{ "$oid" : "0000000000000000fa853239"} , \{ "$oid" : "0000000000000000fa853238"}]}} and Projection is \{ "lrec20" : 1 , "lrec30" : 1 , "lrec35" : 1 , "lrec36" : 1 , "lrec38" : 1 , "lrec65" : 1 , "lrec67" : 1 , "lrec80" : 1 , "lrec90" : 1 , "lrecA0" : 1 , "lrecDC" : 1} in collection: LF31AA
2018-08-26 12:32:56,653 [pool-1-thread-19] DEBUG org.mongodb.driver.protocol.query () Sending query of namespace tpfdf.LF31AA on connection [connectionId\{localValue:12, serverValue:0}] to server localhost:27017
2018-08-26 12:32:58,888 [pool-1-thread-19] DEBUG org.mongodb.driver.protocol.query () Query completed
2018-08-26 12:32:58,891 [pool-1-thread-19] DEBUG org.mongodb.driver.protocol.getmore () Getting more documents from namespace tpfdf.LF31AA with cursor 1 on connection [connectionId\{localValue:14, serverValue:0}] to server localhost:27017

Generated at Thu Feb 08 08:58:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.