[SERVER-1637] Mongodb will never starts listening Created: 18/Aug/10  Updated: 30/Mar/12  Resolved: 18/Aug/10

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 1.6.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Pierre Baillet Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: Linux
Participants:

 Description   

On a first run on a server, mongodb never starts listening on its TCP port. It's likely due to the something related to the initialisation tasks because at the second runs, it works ok.

[16:58][testing] root@f01:/etc/sv# chpst -u nobody /opt/mongodb/bin/mongod --shardsvr --replSet pink-alpha/f01:27103,w01:27103 --dbpath /md0/mongodb-shard-4-pink-alpha --port 27103 --nocursors
Wed Aug 18 16:59:33 MongoDB starting : pid=26069 port=27103 dbpath=/md0/mongodb-shard-4-pink-alpha 64-bit
Wed Aug 18 16:59:33 db version v1.6.1, pdfile version 4.5
Wed Aug 18 16:59:33 git version: nogitversion
Wed Aug 18 16:59:33 sys info: Linux f01 2.6.31-302-ec2 #7-Ubuntu SMP Tue Oct 13 19:55:22 UTC 2009 x86_64 BOOST_LIB_VERSION=1_38
Wed Aug 18 16:59:33 [initandlisten] waiting for connections on port 27103
Wed Aug 18 16:59:33 [initandlisten] ******
Wed Aug 18 16:59:33 [initandlisten] creating replication oplog of size: 1540MB... (use --oplogSize to change)
Wed Aug 18 16:59:33 allocating new datafile /md0/mongodb-shard-4-pink-alpha/local.ns, filling with zeroes...
Wed Aug 18 16:59:33 done allocating datafile /md0/mongodb-shard-4-pink-alpha/local.ns, size: 16MB, took 0.038 secs
Wed Aug 18 16:59:33 allocating new datafile /md0/mongodb-shard-4-pink-alpha/local.0, filling with zeroes...
Wed Aug 18 16:59:33 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 16:59:33 done allocating datafile /md0/mongodb-shard-4-pink-alpha/local.0, size: 64MB, took 0.193 secs
Wed Aug 18 16:59:33 allocating new datafile /md0/mongodb-shard-4-pink-alpha/local.1, filling with zeroes...
Wed Aug 18 16:59:34 done allocating datafile /md0/mongodb-shard-4-pink-alpha/local.1, size: 128MB, took 0.393 secs
Wed Aug 18 16:59:34 allocating new datafile /md0/mongodb-shard-4-pink-alpha/local.2, filling with zeroes...
Wed Aug 18 16:59:43 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 16:59:53 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:03 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:13 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:23 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:33 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:43 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:00:53 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:03 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:13 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:23 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:33 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:43 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:01:53 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:03 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:13 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:23 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:33 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:39 done allocating datafile /md0/mongodb-shard-4-pink-alpha/local.2, size: 2047MB, took 185.032 secs
Wed Aug 18 17:02:43 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:02:53 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:03:03 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)

At the same time:

[17:03][testing] root@f01:~# mongo --port 27103
MongoDB shell version: 1.6.1
connecting to: 127.0.0.1:27103/test
Wed Aug 18 17:04:32 Error: couldn't connect to server 127.0.0.1:27103} (anon):1139
exception: connect failed

Also, when in this state, the daemon cannot be killed quickly and takes some time to shutdown:

^CWed Aug 18 17:04:43 got kill or ctrl c or hup signal 2 (Interrupt), will terminate after current cmd ends
Wed Aug 18 17:04:44 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:04:54 [startReplSets] replSet can't get local.system.replset config from self or any seed (yet)
Wed Aug 18 17:04:59 [initandlisten] ******
Wed Aug 18 17:04:59 [websvr] web admin interface listening on port 28103
Wed Aug 18 17:04:59 [interruptThread] now exiting
Wed Aug 18 17:04:59 dbexit:

Wed Aug 18 17:04:59 [interruptThread] shutdown: going to close listening sockets...
Wed Aug 18 17:04:59 [interruptThread] going to close listening socket: 10
Wed Aug 18 17:04:59 [interruptThread] going to close listening socket: 11
Wed Aug 18 17:04:59 [interruptThread] shutdown: going to flush oplog...
Wed Aug 18 17:04:59 [interruptThread] shutdown: going to close sockets...
Wed Aug 18 17:04:59 [interruptThread] shutdown: waiting for fs preallocator...
Wed Aug 18 17:04:59 [interruptThread] shutdown: closing all files...
Wed Aug 18 17:04:59 closeAllFiles() finished

Wed Aug 18 17:04:59 [interruptThread] shutdown: removing fs lock...
Wed Aug 18 17:04:59 dbexit: really exiting now

Finally, note that when the control-C happens while it's still filling with zeroes, mongo might never stops, unless a KILL -9 is sent.



 Comments   
Comment by Pierre Baillet [ 18/Mar/11 ]

Indeed using XFS fixes the issue and mongo starts up much much faster.

Comment by Eliot Horowitz (Inactive) [ 18/Aug/10 ]

please comment if you still see the problem

Comment by Eliot Horowitz (Inactive) [ 18/Aug/10 ]

I just don't think you waited long enough...
It takes a few minutes to allocate the oplog
You can set the oplog smaller, or use a better filesystem (ext4 or xfs)

Generated at Thu Feb 08 02:57:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.