[SERVER-6734] Mongod give error "too many opens file" Created: 08/Aug/12  Updated: 02/Mar/17  Resolved: 08/Aug/12

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.0.3
Fix Version/s: None

Type: Bug Priority: Blocker - P1
Reporter: jitendra Assignee: Barrie Segal
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux OEL


Issue Links:
Duplicate
is duplicated by SERVER-6987 Mongod give error "too many opens file" Closed
Operating System: Linux
Participants:

 Description   

hi,

We have following setup:
mongos :1
config server :3
mongod (shards):10

all shards are journal enabled.

Mongod1 become non-responsive. Logs show error "too many opens files".
At same time mongos also become non-responsive. The resident memory utilization of mongod1 was 5.5GB.

Below are the logs of that time:
Wed Aug 8 15:10:15 [initandlisten] Listener: accept() returns -1 errno:24 Too many open files
Wed Aug 8 15:10:15 [initandlisten] ERROR: Out of file descriptors. Waiting one second before trying to accept more connections.
Wed Aug 8 15:10:16 [initandlisten] Listener: accept() returns -1 errno:24 Too many open files
Wed Aug 8 15:10:16 [initandlisten] ERROR: Out of file descriptors. Waiting one second before trying to accept more connections.
Wed Aug 8 15:10:17 [initandlisten] Listener: accept() returns -1 errno:24 Too many open files
Wed Aug 8 15:10:17 [initandlisten] ERROR: Out of file descriptors. Waiting one second before trying to accept more connections.
Wed Aug 8 15:10:18 [initandlisten] Listener: accept() returns -1 errno:24 Too many open files

Wed Aug 8 15:19:10 [conn11] Accessing: 00070613 for the first time
Wed Aug 8 15:19:10 [conn11] couldn't open /u01/shard1/00070613.0 errno:24 Too many open files
Wed Aug 8 15:19:10 [conn11] User Assertion: 10085:can't map file memory
Wed Aug 8 15:19:10 [conn8] Accessing: 00070613 for the first time
Wed Aug 8 15:19:10 [conn11] command admin.$cmd command:

{ listDatabases: 1.0 }

ntoreturn:1 exception: can't map file memory code:10085 reslen:126 138744ms
Wed Aug 8 15:19:10 [conn11] Socket recv() conn closed? 192.168.11.103:33821
Wed Aug 8 15:19:10 [conn11] SocketException: remote: 192.168.11.103:33821 error: 9001 socket exception [0] server [192.168.11.103:33821]
Wed Aug 8 15:19:10 [conn8] couldn't open /u01/shard1/00070613.1 errno:24 Too many open files
Wed Aug 8 15:19:10 [conn8] User Assertion: 10085:can't map file memory
Wed Aug 8 15:19:10 [conn8] command admin.$cmd command:

{ listDatabases: 1 }

ntoreturn:1 exception: can't map file memory code:10085 reslen:126 159834ms
Wed Aug 8 15:19:10 [conn8] Socket recv() conn closed? 192.168.11.103:33434

Please let us know of any solution.

Thanks,
Jitendra Verma



 Comments   
Comment by Barrie Segal [ 08/Aug/12 ]

Jitendra,

Can you please post on the mongodb user group so we can troubleshoot your issue there? Thanks.

Barrie

Comment by jitendra [ 08/Aug/12 ]

We did try above things. Although our file descriptor setting was low(Default: 1024). We increased it and ran mongo instances again.
Now when we issue "show dbs" from mongod1, it returns output properly.
But issuing same command from mongos returns failure, "assertion, /cant map file memory".

Another problem we are facing is: MongoD processes taking very high resident memory (5.5GB to 6.0 GB)
Why mongod's taking so much memory?

Thanks,
Jitendra Verma

Comment by Scott Hernandez (Inactive) [ 08/Aug/12 ]

Please see the docs here : http://www.mongodb.org/display/DOCS/Too+Many+Open+Files

Generated at Thu Feb 08 03:12:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.