[SERVER-6230] intermittent connection refused errors in app opening new connections Created: 28/Jun/12  Updated: 24/Jul/12  Resolved: 05/Jul/12

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.1.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ryan Nitz Assignee: Ryan Nitz
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Darwin Rnz4c.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 i386


Operating System: ALL
Participants:

 Description   

I am seeing intermittent connection refused errors from the MMS agent when running locally against 2.1.2.

This was in the mongo logs at the time of the failure:

Wed Jun 27 20:57:32 [conn271] end connection 192.168.0.59:53259 (44 connections now open)
Wed Jun 27 20:58:10 [journal] info journal _rotate called inside dbMutex - ok but should be somewhat rare
Wed Jun 27 20:58:28 [conn34] info DFM::findAll(): extent 0:5000 was empty, skipping ahead. ns:alerts.tmp.alertJobs
Wed Jun 27 20:58:30 [initandlisten] connection accepted from 192.168.0.59:53639 #272 (45 connections now open)
Wed Jun 27 20:58:30 [initandlisten] connection accepted from 192.168.0.59:53640 #273 (46 connections now open)
Wed Jun 27 20:58:33 [conn272] end connection 192.168.0.59:53639 (45 connections now open)
Wed Jun 27 20:58:33 [initandlisten] connection accepted from 192.168.0.59:53645 #274 (46 connections now open)
Wed Jun 27 20:58:37 [journal] info journal _rotate called inside dbMutex - ok but should be somewhat rare

An app exception example (same error when ran with pymongo 2.2):

[exec] 2012-06-27 20:58:28,863 WARNING Problem collecting non-blocking data from (check if it is up and DNS): some.local:27017 - Traceback (most recent call last):
[exec] File "/Users/who/work/dev/mms/server/build/agent-test/nonBlockingStats.py", line 51, in run
[exec] self.monitorConn = self.mmsAgent.getDbConnection( self.hostDef )
[exec] File "/Users/who/work/dev/mms/server/build/agent-test/mmsAgent.py", line 85, in getDbConnection
[exec] return pymongo.Connection( hostDef['mongoUri'] , slave_okay=True )
[exec] File "/Library/Python/2.7/site-packages/pymongo-2.1.1-py2.7-macosx-10.7-intel.egg/pymongo/connection.py", line 381, in _init_
[exec] self.__find_node()
[exec] File "/Library/Python/2.7/site-packages/pymongo-2.1.1-py2.7-macosx-10.7-intel.egg/pymongo/connection.py", line 659, in __find_node
[exec] raise AutoReconnect(', '.join(errors))
[exec] AutoReconnect: could not connect to Rnz4c.local:27017: [Errno 61] Connection refused



 Comments   
Comment by Ryan Nitz [ 05/Jul/12 ]

This appears to be a problem hitting some max/limit for the user on OS X. Despite the setting being high, something else is happening locally. I can still open new connections to mongo via the shell, but something funny happens with DNS resolution.

Comment by Ryan Nitz [ 03/Jul/12 ]

I am having trouble recreating this in a standalone script. I have enabled verbose logging in the db and will post my findings.

Comment by Daniel Pasette (Inactive) [ 02/Jul/12 ]

@ryan, can you please provide any other details on how to repro and your setup?

Generated at Thu Feb 08 03:11:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.