Thu May 26 09:25:49 [conn3633] query: admin.$cmd{ _recvChunkStart: "doctrine.visitor", from: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, configServer: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:25:49 [conn3633] run command admin.$cmd { _recvChunkStart: "doctrine.visitor", from: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, configServer: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:25:49 [conn3633] command: { _recvChunkStart: "doctrine.visitor", from: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, configServer: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:25:49 [conn3633] query admin.$cmd ntoreturn:1 command: { _recvChunkStart: "doctrine.visitor", from: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, configServer: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:63 0ms
Thu May 26 09:25:49 BackgroundJob starting: ConnectBG
Thu May 26 09:25:49 [migrateThread] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:25:49 BackgroundJob starting: ConnectBG
Thu May 26 09:25:49 [migrateThread] updated set (NA) to: NA/na1.applifier.info:27017,na3.applifier.info:27017
Thu May 26 09:25:49 BackgroundJob starting: ConnectBG
Thu May 26 09:25:49 [migrateThread] updated set (NA) to: NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017
Thu May 26 09:25:49 [migrateThread] _check : NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017
Thu May 26 09:25:49 BackgroundJob starting: ReplicaSetMonitorWatcher
Thu May 26 09:25:49 [ReplicaSetMonitorWatcher] starting
Thu May 26 09:25:49 [migrateThread] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:25:49 BackgroundJob starting: ConnectBG
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [conn3633] query: admin.$cmd{ _recvChunkStatus: 1 }
Thu May 26 09:26:21 [conn3633] run command admin.$cmd { _recvChunkStatus: 1 }
Thu May 26 09:26:21 [conn3633] command: { _recvChunkStatus: 1 }
Thu May 26 09:26:21 [conn3633] query admin.$cmd ntoreturn:1 command: { _recvChunkStatus: 1 } reslen:353 0ms
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:21 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [conn3633] query: admin.$cmd{ _recvChunkStatus: 1 }
Thu May 26 09:26:22 [conn3633] run command admin.$cmd { _recvChunkStatus: 1 }
Thu May 26 09:26:22 [conn3633] command: { _recvChunkStatus: 1 }
Thu May 26 09:26:22 [conn3633] query admin.$cmd ntoreturn:1 command: { _recvChunkStatus: 1 } reslen:353 0ms
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:22 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [migrateThread] warning: secondaries having hard time keeping up with migrate
Thu May 26 09:26:23 [conn3633] query: admin.$cmd{ _recvChunkStatus: 1 }
Thu May 26 09:26:23 [conn3633] run command admin.$cmd { _recvChunkStatus: 1 }
Thu May 26 09:26:23 [conn3633] command: { _recvChunkStatus: 1 }
Thu May 26 09:26:23 [conn3633] query admin.$cmd ntoreturn:1 command: { _recvChunkStatus: 1 } reslen:352 0ms
Thu May 26 09:26:23 [conn3633] query: admin.$cmd{ _recvChunkCommit: 1 }
Thu May 26 09:26:23 [conn3633] run command admin.$cmd { _recvChunkCommit: 1 }
Thu May 26 09:26:23 [conn3633] command: { _recvChunkCommit: 1 }
Thu May 26 09:26:23 [migrateThread] migrate commit succeeded flushing to secondaries for 'doctrine.visitor' { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') } -> { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }
Thu May 26 09:26:23 [migrateThread] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:26:23-0", server: "domU-12-31-39-16-15-F8", clientAddr: "(NONE)", time: new Date(1306401983856), what: "moveChunk.to", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 29, step2: 11214, step3: 18066, step4: 4564, step5: 438 } }
Thu May 26 09:26:23 [conn3633] query admin.$cmd ntoreturn:1 command: { _recvChunkCommit: 1 } reslen:350 29ms
Thu May 26 09:26:23 [migrateThread] couldn't create changelog (like race condition): 13105 write $cmd failed on a node: { "errmsg" : "collection already exists", "ok" : 0 } cfg1.applifier.info:30000 ns: config.$cmd cmd: { create: "changelog", size: 10485760, capped: true }
Thu May 26 09:26:29 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:26:29 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:26:49 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:26:49 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:26:49 [conn3664] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:26:49 [conn3664] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:26:49 [conn3664] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:26:49 BackgroundJob starting: ConnectBG
Thu May 26 09:26:49 [conn3664] ReplicaSetMonitor::_checkConnection: eu2.applifier.info:27017 { setName: "EU", ismaster: false, secondary: true, hosts: [ "eu2.applifier.info:27017", "eu3.applifier.info:27017", "eu1.applifier.info:27017" ], primary: "eu1.applifier.info:27017", maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:26:49 BackgroundJob starting: ConnectBG
Thu May 26 09:26:49 [conn3664] updated set (EU) to: EU/eu2.applifier.info:27017,eu3.applifier.info:27017
Thu May 26 09:26:49 BackgroundJob starting: ConnectBG
Thu May 26 09:26:49 [conn3664] updated set (EU) to: EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017
Thu May 26 09:26:49 [conn3664] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:26:49 [LockPinger] creating dist lock ping thread for: cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000
Thu May 26 09:26:49 [LockPinger] dist_lock about to ping for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:26:49 [conn3664] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3664:365467791", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402009969), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1cd97c74617a1f71d9c2') }
Thu May 26 09:26:50 [conn3664] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1cd97c74617a1f71d9c2'), when: new Date(1306402009969), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3664:365467791", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:26:50 [conn3664] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:26:50-1", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:39626", time: new Date(1306402010077), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:26:50 [LockPinger] dist_lock pinged successfully for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:26:50 [conn3664] moveChunk request accepted at version 127|0
Thu May 26 09:26:50 [conn3664] moveChunk number of documents: 83886
Thu May 26 09:26:50 [conn3664] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1cd97c74617a1f71d9c2'), when: new Date(1306402009969), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3664:365467791", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:26:50 [conn3664] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:26:50-2", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:39626", time: new Date(1306402010389), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 60, step2: 175, note: "aborted" } }
Thu May 26 09:26:50 [conn3664] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 536ms
Thu May 26 09:27:03 [conn6217] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:03 [conn6217] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:03 [conn6217] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:03 [conn6217] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:03 [conn6217] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6217:1693546072", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402023802), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1ce77c74617a1f71d9c3') }
Thu May 26 09:27:03 [conn6217] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1ce77c74617a1f71d9c3'), when: new Date(1306402023802), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6217:1693546072", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:03 [conn6217] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:03-3", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38718", time: new Date(1306402023822), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:03 [conn6217] moveChunk request accepted at version 127|0
Thu May 26 09:27:04 [conn6217] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1ce77c74617a1f71d9c3'), when: new Date(1306402023802), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6217:1693546072", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:04 [conn6217] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:04-4", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38718", time: new Date(1306402024207), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 98, note: "aborted" } }
Thu May 26 09:27:04 [conn6217] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 435ms
Thu May 26 09:27:05 [DataFileSync] flushing mmap took 44428ms  for 56 files
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] checking replica set: EU
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] _check : EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: eu2.applifier.info:27017 { setName: "EU", ismaster: false, secondary: true, hosts: [ "eu2.applifier.info:27017", "eu3.applifier.info:27017", "eu1.applifier.info:27017" ], primary: "eu1.applifier.info:27017", maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: eu1.applifier.info:27017 { setName: "EU", ismaster: true, secondary: false, hosts: [ "eu1.applifier.info:27017", "eu3.applifier.info:27017", "eu2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:27:09 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:14 [conn6542] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:14 [conn6542] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:14 [conn6542] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:14 [conn6542] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:14 [conn6542] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6542:1232585025", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402034462), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1cf27c74617a1f71d9c4') }
Thu May 26 09:27:14 [conn6542] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1cf27c74617a1f71d9c4'), when: new Date(1306402034462), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6542:1232585025", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:14 [conn6542] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:14-5", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38808", time: new Date(1306402034492), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:14 [conn6542] moveChunk request accepted at version 127|0
Thu May 26 09:27:14 [conn6542] moveChunk number of documents: 83886
Thu May 26 09:27:14 [conn6542] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1cf27c74617a1f71d9c4'), when: new Date(1306402034462), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6542:1232585025", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:14 [conn6542] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:14-6", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38808", time: new Date(1306402034738), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 70, note: "aborted" } }
Thu May 26 09:27:14 [conn6542] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 301ms
Thu May 26 09:27:17 [conn4280] query: doctrine.view{ _id: ObjectId('4cf141adfb9ad955136e0100') }
Thu May 26 09:27:17 [conn4280] query doctrine.view ntoreturn:1 idhack  reslen:36 0ms
Thu May 26 09:27:20 [LockPinger] dist_lock about to ping for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:27:20 [LockPinger] dist_lock pinged successfully for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:27:24 [DataFileSync] flushing mmap took 3080ms  for 56 files
Thu May 26 09:27:24 [conn6791] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:24 [conn6791] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:24 [conn6791] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:24 [conn6791] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:24 [conn6791] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6791:594964059", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402044975), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1cfc7c74617a1f71d9c5') }
Thu May 26 09:27:25 [conn6791] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1cfc7c74617a1f71d9c5'), when: new Date(1306402044975), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6791:594964059", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:25 [conn6791] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:25-7", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38882", time: new Date(1306402045014), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:25 [conn6791] moveChunk request accepted at version 127|0
Thu May 26 09:27:25 [conn6791] moveChunk number of documents: 83886
Thu May 26 09:27:25 [conn6791] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1cfc7c74617a1f71d9c5'), when: new Date(1306402044975), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn6791:594964059", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:25 [conn6791] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:25-8", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:38882", time: new Date(1306402045345), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 101, note: "aborted" } }
Thu May 26 09:27:25 [conn6791] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 403ms
Thu May 26 09:27:26 [conn6528] query: doctrine.trending_applications{ week_count: { $gt: 10000 } }
Thu May 26 09:27:26 [conn6528] query doctrine.trending_applications reslen:6230 nreturned:53 0ms
Thu May 26 09:27:29 [ReplicaSetMonitorWatcher] checking replica set: EU
Thu May 26 09:27:29 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: eu1.applifier.info:27017 { setName: "EU", ismaster: true, secondary: false, hosts: [ "eu1.applifier.info:27017", "eu3.applifier.info:27017", "eu2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:29 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:27:29 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:35 [conn4315] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:35 [conn4315] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:35 [conn4315] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:35 [conn4315] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:35 [conn4315] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4315:2147038856", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402055699), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1d077c74617a1f71d9c6') }
Thu May 26 09:27:35 [conn4315] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1d077c74617a1f71d9c6'), when: new Date(1306402055699), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4315:2147038856", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:35 [conn4315] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:35-9", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:37458", time: new Date(1306402055714), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:35 [conn4315] moveChunk request accepted at version 127|0
Thu May 26 09:27:35 [conn4315] moveChunk number of documents: 83886
Thu May 26 09:27:35 [conn4315] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1d077c74617a1f71d9c6'), when: new Date(1306402055699), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4315:2147038856", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:35 [conn4315] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:35-10", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:37458", time: new Date(1306402055963), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 52, note: "aborted" } }
Thu May 26 09:27:36 [conn4315] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 309ms
Thu May 26 09:27:38 [conn6405] query: admin.$cmd{ splitVector: "doctrine.visitor", keyPattern: { cn: 1.0, tid: 1.0 }, min: { cn: "WW", tid: ObjectId('4dae3e6a7eef113e35001b85') }, max: { cn: "WW", tid: ObjectId('4db942f0c47a101a60001a56') }, maxChunkSizeBytes: 67108864, maxSplitPoints: 2, maxChunkObjects: 250000 }
Thu May 26 09:27:38 [conn6405] run command admin.$cmd { splitVector: "doctrine.visitor", keyPattern: { cn: 1.0, tid: 1.0 }, min: { cn: "WW", tid: ObjectId('4dae3e6a7eef113e35001b85') }, max: { cn: "WW", tid: ObjectId('4db942f0c47a101a60001a56') }, maxChunkSizeBytes: 67108864, maxSplitPoints: 2, maxChunkObjects: 250000 }
Thu May 26 09:27:38 [conn6405] request split points lookup for chunk doctrine.visitor { : "WW", : ObjectId('4dae3e6a7eef113e35001b85') } -->> { : "WW", : ObjectId('4db942f0c47a101a60001a56') }
Thu May 26 09:27:38 [conn6405] picked a split key: { cn: "WW", tid: ObjectId('4db82735f301f1376b000aec') }
Thu May 26 09:27:38 [conn6405] warning: Finding the split vector for doctrine.visitor over { cn: 1.0, tid: 1.0 } keyCount: 90200 numSplits: 1 lookedAt: 28902 took 157ms
Thu May 26 09:27:38 [conn6405] query admin.$cmd ntoreturn:1 command: { splitVector: "doctrine.visitor", keyPattern: { cn: 1.0, tid: 1.0 }, min: { cn: "WW", tid: ObjectId('4dae3e6a7eef113e35001b85') }, max: { cn: "WW", tid: ObjectId('4db942f0c47a101a60001a56') }, maxChunkSizeBytes: 67108864, maxSplitPoints: 2, maxChunkObjects: 250000 } reslen:105 157ms
Thu May 26 09:27:46 [conn4045] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:46 [conn4045] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:46 [conn4045] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:46 [conn4045] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:46 [conn4045] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402066258), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1d127c74617a1f71d9c7') }
Thu May 26 09:27:46 [conn4045] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1d127c74617a1f71d9c7'), when: new Date(1306402066258), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:46 [conn4045] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:46-11", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:36562", time: new Date(1306402066275), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:46 [conn4045] moveChunk request accepted at version 127|0
Thu May 26 09:27:46 [conn4045] moveChunk number of documents: 83886
Thu May 26 09:27:46 [conn4045] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1d127c74617a1f71d9c7'), when: new Date(1306402066258), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:46 [conn4045] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:46-12", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:36562", time: new Date(1306402066459), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 53, note: "aborted" } }
Thu May 26 09:27:46 [conn4045] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 225ms
Thu May 26 09:27:49 [ReplicaSetMonitorWatcher] checking replica set: EU
Thu May 26 09:27:49 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: eu1.applifier.info:27017 { setName: "EU", ismaster: true, secondary: false, hosts: [ "eu1.applifier.info:27017", "eu3.applifier.info:27017", "eu2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:49 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:27:49 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:27:50 [LockPinger] dist_lock about to ping for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:27:50 [LockPinger] dist_lock pinged successfully for: domU-12-31-39-16-15-F8:1306402009:1557774390
Thu May 26 09:27:56 [conn4045] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:56 [conn4045] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:56 [conn4045] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:56 [conn4045] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:27:56 [conn4045] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402076724), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1d1c7c74617a1f71d9c8') }
Thu May 26 09:27:56 [conn4045] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1d1c7c74617a1f71d9c8'), when: new Date(1306402076724), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:56 [conn4045] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:56-13", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:36562", time: new Date(1306402076739), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:27:56 [conn4045] moveChunk request accepted at version 127|0
Thu May 26 09:27:57 [conn4045] moveChunk number of documents: 83886
Thu May 26 09:27:57 [conn4045] dist_lock unlock: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 0, ts: ObjectId('4dde1d1c7c74617a1f71d9c8'), when: new Date(1306402076724), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn4045:1237633890", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:27:57 [conn4045] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:27:57-14", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:36562", time: new Date(1306402077071), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, step2: 67, note: "aborted" } }
Thu May 26 09:27:57 [conn4045] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:357 381ms
Thu May 26 09:28:07 [conn3653] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:07 [conn3653] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:07 [conn3653] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:07 [conn3653] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:07 [conn3653] dist_lock about to aquire lock: { state: 1, who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3653:290593812", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", when: new Date(1306402087654), why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }", ts: ObjectId('4dde1d277c74617a1f71d9c9') }
Thu May 26 09:28:07 [conn3653] dist_lock lock gotLock: 1 now: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1d277c74617a1f71d9c9'), when: new Date(1306402087654), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3653:290593812", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" }
Thu May 26 09:28:07 [conn3653] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:28:07-15", server: "domU-12-31-39-16-15-F8", clientAddr: "10.100.187.118:39614", time: new Date(1306402087707), what: "moveChunk.start", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, from: "EU", to: "NA" } }
Thu May 26 09:28:07 [conn3653] moveChunk request accepted at version 127|0
Thu May 26 09:28:09 [ReplicaSetMonitorWatcher] checking replica set: EU
Thu May 26 09:28:09 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: eu1.applifier.info:27017 { setName: "EU", ismaster: true, secondary: false, hosts: [ "eu1.applifier.info:27017", "eu3.applifier.info:27017", "eu2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:28:09 [ReplicaSetMonitorWatcher] checking replica set: NA
Thu May 26 09:28:09 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: na1.applifier.info:27017 { setName: "NA", ismaster: true, secondary: false, hosts: [ "na1.applifier.info:27017", "na3.applifier.info:27017", "na2.applifier.info:27017" ], maxBsonObjectSize: 16777216, ok: 1.0 }
Thu May 26 09:28:10 [conn7384] query: admin.$cmd{ moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:10 [conn7384] run command admin.$cmd { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:10 [conn7384] command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:10 [conn7384] received moveChunk request: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" }
Thu May 26 09:28:10 [conn7384] dist_lock lock failed because taken by: { _id: "doctrine.visitor", process: "domU-12-31-39-16-15-F8:1306402009:1557774390", state: 1, ts: ObjectId('4dde1d277c74617a1f71d9c9'), when: new Date(1306402087654), who: "domU-12-31-39-16-15-F8:1306402009:1557774390:conn3653:290593812", why: "migrate-{ cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }" } elapsed minutes: 0
Thu May 26 09:28:10 [conn7384] about to log metadata event: { _id: "domU-12-31-39-16-15-F8-2011-05-26T09:28:10-16", server: "domU-12-31-39-16-15-F8", clientAddr: "10.78.61.110:42259", time: new Date(1306402090331), what: "moveChunk.from", ns: "doctrine.visitor", details: { min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, step1: 0, note: "aborted" } }
Thu May 26 09:28:10 [conn7384] query admin.$cmd ntoreturn:1 command: { moveChunk: "doctrine.visitor", from: "EU/eu2.applifier.info:27017,eu3.applifier.info:27017,eu1.applifier.info:27017", to: "NA/na1.applifier.info:27017,na3.applifier.info:27017,na2.applifier.info:27017", min: { cn: "EU", tid: ObjectId('4dc57d050e4208ce2a00006c') }, max: { cn: "EU", tid: ObjectId('4dc594607eb394a15c000dcc') }, maxChunkSizeBytes: 67108864, shardId: "doctrine.visitor-cn_"EU"tid_ObjectId('4dc57d050e4208ce2a00006c')", configdb: "cfg1.applifier.info:30000,cfg2.applifier.info:30000,cfg3.applifier.info:30000" } reslen:386 36ms

--- After this only lines are from connections opening, making queries but queries do not return and mongo goes unresponsive