Wed Oct 3 16:43:10 warning: running with 1 config server should be done only for testing purposes and is not recommended for production Wed Oct 3 16:43:10 [mongosMain] MongoS version 2.2.1-rc1-pre- starting: pid=8417 port=27017 64-bit host=mongos (--help for usage) Wed Oct 3 16:43:10 [mongosMain] git version: f87923d19fc3d5b65d4c1743b758dd6bc03b783d Wed Oct 3 16:43:10 [mongosMain] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49 Wed Oct 3 16:43:10 [mongosMain] options: { configdb: "mongoconf:27018", vvvvv: true } Wed Oct 3 16:43:10 [mongosMain] config string : mongoconf:27018 Wed Oct 3 16:43:10 [mongosMain] creating new connection to:mongoconf:27018 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [mongosMain] connected connection! Wed Oct 3 16:43:10 BackgroundJob starting: CheckConfigServers Wed Oct 3 16:43:10 [CheckConfigServers] creating new connection to:mongoconf:27018 Wed Oct 3 16:43:10 [mongosMain] MaxChunkSize: 64 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [mongosMain] fd limit hard:4096 soft:1024 max conn: 819 Wed Oct 3 16:43:10 BackgroundJob starting: Balancer Wed Oct 3 16:43:10 [Balancer] about to contact config servers and shards Wed Oct 3 16:43:10 [Balancer] creating new connection to:mongoconf:27018 Wed Oct 3 16:43:10 [mongosMain] waiting for connections on port 27017 Wed Oct 3 16:43:10 [websvr] fd limit hard:4096 soft:1024 max conn: 819 Wed Oct 3 16:43:10 [websvr] admin web console waiting for connections on port 28017 Wed Oct 3 16:43:10 [CheckConfigServers] connected connection! Wed Oct 3 16:43:10 BackgroundJob starting: PeriodicTask::Runner Wed Oct 3 16:43:10 BackgroundJob starting: cursorTimeout Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] connected connection! Wed Oct 3 16:43:10 [Balancer] starting new replica set monitor for replica set rs1 with seed of mongo1:27017,mongo2:27017 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] successfully connected to seed mongo1:27017 for replica set rs1 Wed Oct 3 16:43:10 [Balancer] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275390841), ok: 1.0 } Wed Oct 3 16:43:10 [Balancer] changing hosts to { 0: "mongo1:27017", 1: "mongo2:27017" } from rs1/ Wed Oct 3 16:43:10 [Balancer] trying to add new host mongo1:27017 to replica set rs1 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] successfully connected to new host mongo1:27017 in replica set rs1 Wed Oct 3 16:43:10 [Balancer] trying to add new host mongo2:27017 to replica set rs1 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] warning: cannot connect to new host mongo2:27017 to replica set rs1 :: caused by :: 15927 couldn't connect to server mongo2:27017 Wed Oct 3 16:43:10 [Balancer] creating new connection to:mongo1:27017 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] connected connection! Wed Oct 3 16:43:10 [Balancer] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:10 [Balancer] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:10 [Balancer] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:10 [Balancer] _check : rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:10 [Balancer] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275390844), ok: 1.0 } Wed Oct 3 16:43:10 [Balancer] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:10 [Balancer] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:10 [Balancer] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:10 [Balancer] trying reconnect to mongo2:27017 Wed Oct 3 16:43:10 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:10 [Balancer] reconnect mongo2:27017 failed couldn't connect to server mongo2:27017 Wed Oct 3 16:43:10 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [CONNECT_ERROR] for mongo2:27017 Wed Oct 3 16:43:11 [Balancer] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275391845), ok: 1.0 } Wed Oct 3 16:43:11 [Balancer] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:11 [Balancer] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:11 [Balancer] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:11 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [FAILED_STATE] for mongo2:27017 Wed Oct 3 16:43:12 [Balancer] warning: No primary detected for set rs1 Wed Oct 3 16:43:12 [Balancer] replica set monitor for replica set rs1 started, address is rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:12 [Balancer] _check : rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:12 BackgroundJob starting: ReplicaSetMonitorWatcher Wed Oct 3 16:43:12 [ReplicaSetMonitorWatcher] starting Wed Oct 3 16:43:12 [Balancer] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275392847), ok: 1.0 } Wed Oct 3 16:43:12 [Balancer] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:12 [Balancer] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:12 [Balancer] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:12 [Balancer] trying reconnect to mongo2:27017 Wed Oct 3 16:43:12 BackgroundJob starting: ConnectBG Wed Oct 3 16:43:12 [Balancer] reconnect mongo2:27017 failed couldn't connect to server mongo2:27017 Wed Oct 3 16:43:12 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [CONNECT_ERROR] for mongo2:27017 Wed Oct 3 16:43:12 [mongosMain] connection accepted from 10.0.3.1:37830 #1 (1 connection now open) Wed Oct 3 16:43:12 [conn1] DBConfig unserialize: admin { _id: "admin", partitioned: false, primary: "config" } Wed Oct 3 16:43:12 [conn1] found 0 dropped collections and 0 sharded collections for database admin Wed Oct 3 16:43:12 [conn1] Request::process begin ns: admin.$cmd msg id: 1 op: 2004 attempt: 0 Wed Oct 3 16:43:12 [conn1] single query: admin.$cmd { isMaster: 1 } ntoreturn: -1 options : 0 Wed Oct 3 16:43:12 [conn1] Request::process end ns: admin.$cmd msg id: 1 op: 2004 attempt: 0 0ms Wed Oct 3 16:43:12 [conn1] couldn't find database [db] in config db Wed Oct 3 16:43:12 [conn1] _check : rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:12 [conn1] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275392879), ok: 1.0 } Wed Oct 3 16:43:12 [conn1] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:12 [conn1] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:12 [conn1] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:12 [conn1] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [FAILED_STATE] for mongo2:27017 Wed Oct 3 16:43:13 [Balancer] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275393847), ok: 1.0 } Wed Oct 3 16:43:13 [Balancer] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:13 [Balancer] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:13 [Balancer] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:13 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [FAILED_STATE] for mongo2:27017 Wed Oct 3 16:43:13 [conn1] ReplicaSetMonitor::_checkConnection: mongo1:27017 { setName: "rs1", ismaster: false, secondary: true, hosts: [ "mongo1:27017", "mongo2:27017" ], arbiters: [ "mongoarbiter1:27017" ], me: "mongo1:27017", maxBsonObjectSize: 16777216, localTime: new Date(1349275393879), ok: 1.0 } Wed Oct 3 16:43:13 [conn1] dbclient_rs nodes[0].ok = true mongo1:27017 Wed Oct 3 16:43:13 [conn1] dbclient_rs nodes[1].ok = false mongo2:27017 Wed Oct 3 16:43:13 [conn1] dbclient_rs _checkStatus couldn't _find(mongoarbiter1:27017) Wed Oct 3 16:43:13 [conn1] ReplicaSetMonitor::_checkConnection: caught exception mongo2:27017 socket exception [FAILED_STATE] for mongo2:27017 Wed Oct 3 16:43:14 [Balancer] warning: No primary detected for set rs1 Wed Oct 3 16:43:14 [Balancer] User Assertion: 10009:ReplicaSetMonitor no master found for set: rs1 Wed Oct 3 16:43:14 [Balancer] warning: could not initialize balancer, please check that all shards and config servers are up: socket exception [CONNECT_ERROR] for rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:14 [Balancer] will retry to initialize balancer in one minute Wed Oct 3 16:43:14 [conn1] warning: No primary detected for set rs1 Wed Oct 3 16:43:14 [conn1] User Assertion: 10009:ReplicaSetMonitor no master found for set: rs1 Wed Oct 3 16:43:14 [conn1] warning: error creating initial database config information :: caused by :: socket exception [CONNECT_ERROR] for rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:14 [conn1] DBException in process: error creating initial database config information :: caused by :: socket exception [CONNECT_ERROR] for rs1/mongo1:27017,mongo2:27017 Wed Oct 3 16:43:14 [conn1] Socket recv() conn closed? 10.0.3.1:37830 Wed Oct 3 16:43:14 [conn1] SocketException: remote: 10.0.3.1:37830 error: 9001 socket exception [0] server [10.0.3.1:37830] Wed Oct 3 16:43:14 [conn1] end connection 10.0.3.1:37830 (0 connections now open)