Fri Oct 5 23:15:12 got kill or ctrl c or hup signal 15 (Terminated), will terminate after current cmd ends Fri Oct 5 23:15:19 [conn755708] end connection 10.118.249.173:32934 Fri Oct 5 23:15:19 [initandlisten] connection accepted from 10.118.249.173:32936 #755710 Fri Oct 5 23:15:19 [conn755710] authenticate: { authenticate: 1, nonce: "e021af2e5fff9d68", user: "__system", key: "b4684729b0ef9fdf83abd9abab3a83a7" } Fri Oct 5 23:15:20 [conn755709] end connection 10.211.21.71:36000 Fri Oct 5 23:15:20 [initandlisten] connection accepted from 10.211.21.71:36002 #755711 Fri Oct 5 23:15:20 [conn755711] authenticate: { authenticate: 1, nonce: "223895a8c8911b96", user: "__system", key: "c73289360c457df6270b3d0d90d8066e" } ***** SERVER RESTARTED ***** Fri Oct 5 23:15:40 [initandlisten] MongoDB starting : pid=16451 port=27018 dbpath=/var/lib/mongodb 64-bit host=domU-12-31-39-07-B6-7C.compute-1.internal Fri Oct 5 23:15:40 [initandlisten] db version v2.0.4, pdfile version 4.5 Fri Oct 5 23:15:40 [initandlisten] git version: 329f3c47fe8136c03392c8f0e548506cb21f8ebf Fri Oct 5 23:15:40 [initandlisten] build info: Linux ip-10-110-9-236 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41 Fri Oct 5 23:15:40 [initandlisten] options: { auth: "true", command: [ "run" ], config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", keyFile: "/.mongo/mongokey", logappend: "true", logpath: "/var/log/mongodb/mongodb.log", nohttpinterface: "true", port: 27018, replSet: "REDMongo1", shardsvr: "true" } ************** old lock file: /var/lib/mongodb/mongod.lock. probably means unclean shutdown, but there are no journal files to recover. this is likely human error or filesystem corruption. found 10 dbs. see: http://dochub.mongodb.org/core/repair for more information ************* Fri Oct 5 23:15:40 [initandlisten] exception in initAndListen: 12596 old lock file, terminating Fri Oct 5 23:15:40 dbexit: Fri Oct 5 23:15:40 [initandlisten] shutdown: going to close listening sockets... Fri Oct 5 23:15:40 [initandlisten] shutdown: going to flush diaglog... Fri Oct 5 23:15:40 [initandlisten] shutdown: going to close sockets... Fri Oct 5 23:15:40 [initandlisten] shutdown: waiting for fs preallocator... Fri Oct 5 23:15:40 [initandlisten] shutdown: lock for final commit... Fri Oct 5 23:15:40 [initandlisten] shutdown: final commit... Fri Oct 5 23:15:40 [initandlisten] shutdown: closing all files... Fri Oct 5 23:15:40 [initandlisten] closeAllFiles() finished Fri Oct 5 23:15:40 dbexit: really exiting now ***** SERVER RESTARTED ***** Fri Oct 5 23:19:24 [initandlisten] MongoDB starting : pid=16471 port=27018 dbpath=/var/lib/mongodb 64-bit host=domU-12-31-39-07-B6-7C.compute-1.internal Fri Oct 5 23:19:24 [initandlisten] db version v2.0.4, pdfile version 4.5 Fri Oct 5 23:19:24 [initandlisten] git version: 329f3c47fe8136c03392c8f0e548506cb21f8ebf Fri Oct 5 23:19:24 [initandlisten] build info: Linux ip-10-110-9-236 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41 Fri Oct 5 23:19:24 [initandlisten] options: { auth: "true", command: [ "run" ], config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", keyFile: "/.mongo/mongokey", logappend: "true", logpath: "/var/log/mongodb/mongodb.log", nohttpinterface: "true", port: 27018, replSet: "REDMongo1", shardsvr: "true" } Fri Oct 5 23:19:24 [initandlisten] journal dir=/var/lib/mongodb/journal Fri Oct 5 23:19:24 [initandlisten] recover : no journal files present, no recovery needed Fri Oct 5 23:19:24 [initandlisten] preallocateIsFaster=true 2.08 Fri Oct 5 23:19:24 [initandlisten] preallocateIsFaster=true 2.14 Fri Oct 5 23:19:25 [initandlisten] preallocateIsFaster=true 2.38 Fri Oct 5 23:19:25 [initandlisten] preallocating a journal file /var/lib/mongodb/journal/prealloc.0 Fri Oct 5 23:19:49 [initandlisten] preallocating a journal file /var/lib/mongodb/journal/prealloc.1 Fri Oct 5 23:20:13 [initandlisten] preallocating a journal file /var/lib/mongodb/journal/prealloc.2 Fri Oct 5 23:20:38 [initandlisten] waiting for connections on port 27018 Fri Oct 5 23:20:38 [initandlisten] connection accepted from 127.0.0.1:49680 #1 Fri Oct 5 23:20:38 [conn1] authenticate: { authenticate: 1, nonce: "ff7d366975ac7442", user: "__system", key: "4c08278853d3e061cf856123d76efd39" } Fri Oct 5 23:20:38 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG) Fri Oct 5 23:20:38 [rsStart] replSet info you may need to run replSetInitiate -- rs.initiate() in the shell -- if that is not already done Fri Oct 5 23:20:38 [initandlisten] connection accepted from 10.211.21.71:59329 #2 Fri Oct 5 23:20:38 [conn2] authenticate: { authenticate: 1, nonce: "ea2df907bd50e7bd", user: "__system", key: "af2f4868484655c619f5b551644f3faf" } Fri Oct 5 23:20:39 [initandlisten] connection accepted from 10.68.97.53:55363 #3 Fri Oct 5 23:20:39 [initandlisten] connection accepted from 10.118.249.173:55048 #4 Fri Oct 5 23:20:39 [conn4] authenticate: { authenticate: 1, nonce: "6f19d0544511312b", user: "__system", key: "8fd55bfd3eea3cca75afb54e96b494c4" } Fri Oct 5 23:20:41 [initandlisten] connection accepted from 10.248.115.38:47396 #5 Fri Oct 5 23:20:41 [conn5] authenticate: { authenticate: 1, nonce: "438b397a72add534", user: "__system", key: "a1942f611a72271fe63bfb171c6e1de4" } Fri Oct 5 23:20:41 [initandlisten] connection accepted from 10.68.97.53:55364 #6 Fri Oct 5 23:20:41 [conn6] authenticate: { authenticate: 1, nonce: "feab7c4df17006e3", user: "__system", key: "0ae34dc83a94f4bb4d0da455bfdb326e" } Fri Oct 5 23:20:47 [initandlisten] connection accepted from 10.248.115.38:47397 #7 Fri Oct 5 23:20:47 [initandlisten] connection accepted from 10.242.230.204:45760 #8 Fri Oct 5 23:20:48 [rsStart] trying to contact mongos1a.qmobix.net:27018 Fri Oct 5 23:20:48 [rsStart] trying to contact mongos1r1.qmobix.net:27018 Fri Oct 5 23:20:48 [rsStart] replSet got config version 3 from a remote, saving locally Fri Oct 5 23:20:48 [rsStart] replSet info saving a newer config version to local.system.replset Fri Oct 5 23:20:48 [FileAllocator] allocating new datafile /var/lib/mongodb/local.ns, filling with zeroes... Fri Oct 5 23:20:48 [FileAllocator] creating directory /var/lib/mongodb/_tmp Fri Oct 5 23:20:48 [FileAllocator] done allocating datafile /var/lib/mongodb/local.ns, size: 16MB, took 0.039 secs Fri Oct 5 23:20:48 [FileAllocator] allocating new datafile /var/lib/mongodb/local.0, filling with zeroes... Fri Oct 5 23:20:48 [FileAllocator] done allocating datafile /var/lib/mongodb/local.0, size: 64MB, took 0.149 secs Fri Oct 5 23:20:48 [FileAllocator] allocating new datafile /var/lib/mongodb/local.1, filling with zeroes... Fri Oct 5 23:20:50 [rsStart] replSet saveConfigLocally done Fri Oct 5 23:20:50 [FileAllocator] done allocating datafile /var/lib/mongodb/local.1, size: 128MB, took 1.426 secs Fri Oct 5 23:20:50 [rsStart] replSet STARTUP2 Fri Oct 5 23:20:50 [rsSync] ****** Fri Oct 5 23:20:50 [rsSync] creating replication oplog of size: 4534MB... Fri Oct 5 23:20:50 [FileAllocator] allocating new datafile /var/lib/mongodb/local.2, filling with zeroes... Fri Oct 5 23:20:50 [rsHealthPoll] replSet member mongos1r1.qmobix.net:27018 is up Fri Oct 5 23:20:50 [rsHealthPoll] replSet member mongos1r1.qmobix.net:27018 is now in state PRIMARY Fri Oct 5 23:20:50 [rsHealthPoll] replSet member mongos1a.qmobix.net:27018 is up Fri Oct 5 23:20:50 [rsHealthPoll] replSet member mongos1a.qmobix.net:27018 is now in state ARBITER Fri Oct 5 23:20:51 [conn4] end connection 10.118.249.173:55048 Fri Oct 5 23:20:51 [initandlisten] connection accepted from 10.118.249.173:55050 #9 Fri Oct 5 23:20:51 [conn9] authenticate: { authenticate: 1, nonce: "91619f16b290538c", user: "__system", key: "12d1df5f9d8f24b6d3d645e65bbf5a32" } Fri Oct 5 23:20:52 [conn2] end connection 10.211.21.71:59329 Fri Oct 5 23:20:52 [initandlisten] connection accepted from 10.211.21.71:59330 #10 Fri Oct 5 23:20:52 [conn10] authenticate: { authenticate: 1, nonce: "a190d4ee1a129150", user: "__system", key: "cfdc8df1e1830f923fc6baca04cb7ed5" } Fri Oct 5 23:21:21 [conn9] end connection 10.118.249.173:55050 Fri Oct 5 23:21:21 [initandlisten] connection accepted from 10.118.249.173:55052 #11 Fri Oct 5 23:21:21 [conn11] authenticate: { authenticate: 1, nonce: "529878f529011e09", user: "__system", key: "e074a740bcc72ab7ae29ea7ff5f68e9b" } Fri Oct 5 23:21:22 [conn10] end connection 10.211.21.71:59330 Fri Oct 5 23:21:22 [initandlisten] connection accepted from 10.211.21.71:59332 #12 Fri Oct 5 23:21:22 [conn12] authenticate: { authenticate: 1, nonce: "425d642b5f5c4ca5", user: "__system", key: "9cce553714b4c7073f433f726d3137e7" } Fri Oct 5 23:21:25 [FileAllocator] done allocating datafile /var/lib/mongodb/local.2, size: 2047MB, took 35.115 secs Fri Oct 5 23:21:25 [FileAllocator] allocating new datafile /var/lib/mongodb/local.3, filling with zeroes... Fri Oct 5 23:21:51 [conn11] end connection 10.118.249.173:55052 Fri Oct 5 23:21:51 [initandlisten] connection accepted from 10.118.249.173:55054 #13 Fri Oct 5 23:21:51 [conn13] authenticate: { authenticate: 1, nonce: "e26f2ec4aab9d970", user: "__system", key: "ebf17c2393fbbb9852cf370bc3c3619a" } Fri Oct 5 23:21:52 [conn12] end connection 10.211.21.71:59332 Fri Oct 5 23:21:52 [initandlisten] connection accepted from 10.211.21.71:43292 #14 Fri Oct 5 23:21:52 [conn14] authenticate: { authenticate: 1, nonce: "c543794343f4b9ea", user: "__system", key: "18a1d86cdcd5c1c947ccf9bd3d76b20b" } Fri Oct 5 23:22:08 [FileAllocator] done allocating datafile /var/lib/mongodb/local.3, size: 2047MB, took 43.73 secs Fri Oct 5 23:22:08 [FileAllocator] allocating new datafile /var/lib/mongodb/local.4, filling with zeroes... Fri Oct 5 23:22:21 [conn13] end connection 10.118.249.173:55054 Fri Oct 5 23:22:21 [initandlisten] connection accepted from 10.118.249.173:55056 #15 Fri Oct 5 23:22:21 [conn15] authenticate: { authenticate: 1, nonce: "9c74ebd6845c4844", user: "__system", key: "7b572307a0921f8e3c80d2545d783ebe" } Fri Oct 5 23:22:22 [conn14] end connection 10.211.21.71:43292 Fri Oct 5 23:22:22 [initandlisten] connection accepted from 10.211.21.71:43294 #16 Fri Oct 5 23:22:22 [conn16] authenticate: { authenticate: 1, nonce: "dc0ee92829117d0b", user: "__system", key: "e9662ba0d92d0c0e12f7dc3bfaa56aab" } Fri Oct 5 23:22:51 [conn15] end connection 10.118.249.173:55056 Fri Oct 5 23:22:51 [initandlisten] connection accepted from 10.118.249.173:55058 #17 Fri Oct 5 23:22:51 [conn17] authenticate: { authenticate: 1, nonce: "31accaa907279359", user: "__system", key: "a9c2d7b5a599fd98069f269ac2edc748" } Fri Oct 5 23:22:52 [conn16] end connection 10.211.21.71:43294 Fri Oct 5 23:22:52 [initandlisten] connection accepted from 10.211.21.71:43296 #18 Fri Oct 5 23:22:52 [conn18] authenticate: { authenticate: 1, nonce: "cbbce25cda1821e4", user: "__system", key: "78eed854248b1609569d3298e9042081" } Fri Oct 5 23:22:57 [FileAllocator] done allocating datafile /var/lib/mongodb/local.4, size: 2047MB, took 48.322 secs Fri Oct 5 23:23:15 [rsSync] ****** Fri Oct 5 23:23:15 [rsSync] replSet initial sync pending Fri Oct 5 23:23:15 [rsSync] replSet syncing to: mongos1r1.qmobix.net:27018 Fri Oct 5 23:23:15 [rsSync] build index local.me { _id: 1 } Fri Oct 5 23:23:15 [rsSync] build index done 0 records 0 secs Fri Oct 5 23:23:15 [rsSync] replSet initial sync drop all databases Fri Oct 5 23:23:15 [rsSync] dropAllDatabasesExceptLocal 1 Fri Oct 5 23:23:15 [rsSync] replSet initial sync clone all databases