-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Cluster Management
-
Labels:None
We are using replica-set and providing the configuration like the following, but these configuration options does not seem to work. We seem to be getting 2 problems-
1) if all the nodes of the mongo db are down, requests (e.g. mongoTemplate.save) hang for about 60-120 seconds and then throw an exception. Following is the configuration we are using, the behavior is same whether we use mongo options or not. We are using mongo db for logging purpose only which is used for logging requests and responses and want even if all nodes of the mongo db are down, requests should not hang (if these fail, these should exit immediately)
2) We see very fast re-connection attempts being made (hundreds of them in a sec) which clogs the log files very fast and we are not sure if #1 may be caused by this.
<mongo:mongo id="mongo" replica-set="${nosql.db.replicaset}">
<mongo:options
connect-timeout="5000"
max-wait-time="1500"
auto-connect-retry="true"
socket-keep-alive="true"
socket-timeout="1500"
slave-ok="true"
write-number="1"
write-timeout="1000"
write-fsync="true"/>
</mongo:mongo>
The following error is seen in the logs repeatedly-
org.springframework.data.mongodb.UncategorizedMongoDbException: Rare case where master=null, probably all servers are down; nested exception is com.mongodb.MongoException: Rare case where master=null, probably all servers are down
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:78)
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:1694)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:375)
at org.springframework.data.mongodb.core.MongoTemplate.collectionExists(MongoTemplate.java:437)
at com.expedia.globalpayments.svs.checker.MongoDatabaseChecker.check(MongoDatabaseChecker.java:56)
at com.expedia.e3.es.payment.common.status.cache.ComponentStatusUpdateTask.update(ComponentStatusUpdateTask.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:299)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:111)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: com.mongodb.MongoException: Rare case where master=null, probably all servers are down
at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:435)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:277)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:256)
at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:289)
at com.mongodb.DB.command(DB.java:259)
at com.mongodb.DB.command(DB.java:241)
at com.mongodb.DB.command(DB.java:282)
at com.mongodb.DB.command(DB.java:196)
at com.mongodb.DB$AuthenticationCredentials.authenticate(DB.java:796)
at com.mongodb.DB.authenticate(DB.java:569)
at org.springframework.data.mongodb.core.MongoDbUtils.doGetDB(MongoDbUtils.java:114)
at org.springframework.data.mongodb.core.MongoDbUtils.getDB(MongoDbUtils.java:75)
at org.springframework.data.mongodb.core.SimpleMongoDbFactory.getDb(SimpleMongoDbFactory.java:118)
at org.springframework.data.mongodb.core.SimpleMongoDbFactory.getDb(SimpleMongoDbFactory.java:107)
at org.springframework.data.mongodb.core.MongoTemplate.getDb(MongoTemplate.java:1286)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:372)
... 12 more
Any help is highly appreciated-
Thanks,
Atul Gupta