Tue Dec 11 05:49:55 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:49:55 [conn389518] end connection 10.120.13.32:16452 (2740 connections now open) Tue Dec 11 05:49:55 [conn393158] end connection 10.120.13.42:52341 (2739 connections now open) Tue Dec 11 05:49:55 [initandlisten] connection accepted from 10.120.13.42:52366 #393162 (2740 connections now open) Tue Dec 11 05:49:56 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:49:57 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:49:58 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:49:59 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:00 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:01 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:02 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:03 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:04 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:05 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:06 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:07 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:08 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:09 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:10 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:11 [conn393159] end connection 10.121.13.41:6289 (2739 connections now open) Tue Dec 11 05:50:11 [initandlisten] connection accepted from 10.121.13.41:6293 #393163 (2740 connections now open) Tue Dec 11 05:50:11 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:12 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:13 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:14 [conn393160] end connection 10.120.13.42:52363 (2739 connections now open) Tue Dec 11 05:50:14 [initandlisten] connection accepted from 10.120.13.42:52372 #393164 (2740 connections now open) Tue Dec 11 05:50:14 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:15 [conn393161] end connection 10.121.13.42:20736 (2739 connections now open) Tue Dec 11 05:50:15 [initandlisten] connection accepted from 10.121.13.42:20740 #393165 (2740 connections now open) Tue Dec 11 05:50:15 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:16 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "catchup", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:17 [conn370166] moveChunk data transfer progress: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "steady", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 6 Tue Dec 11 05:50:17 [conn370166] moveChunk setting version to: 6|0||000000000000000000000000 Tue Dec 11 05:50:25 [conn393162] end connection 10.120.13.42:52366 (2739 connections now open) Tue Dec 11 05:50:25 [initandlisten] connection accepted from 10.120.13.42:52375 #393166 (2740 connections now open) Tue Dec 11 05:50:42 [conn393163] end connection 10.121.13.41:6293 (2739 connections now open) Tue Dec 11 05:50:42 [initandlisten] connection accepted from 10.121.13.41:6297 #393167 (2740 connections now open) Tue Dec 11 05:50:44 [conn393164] end connection 10.120.13.42:52372 (2739 connections now open) Tue Dec 11 05:50:44 [initandlisten] connection accepted from 10.120.13.42:52380 #393168 (2740 connections now open) Tue Dec 11 05:50:46 [conn393165] end connection 10.121.13.42:20740 (2739 connections now open) Tue Dec 11 05:50:46 [initandlisten] connection accepted from 10.121.13.42:20744 #393169 (2740 connections now open) Tue Dec 11 05:50:48 [conn370166] moveChunk migrate commit not accepted by TO-shard: { active: true, ns: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, shardKeyPattern: { _id: 1 }, state: "fail", errmsg: "", counts: { cloned: 19935, clonedBytes: 8525712, catchup: 0, steady: 0 }, errmsg: "", ok: 0.0 } resetting shard version to: 0|0||000000000000000000000000 Tue Dec 11 05:50:48 [conn370166] moveChunk global lock acquired to reset shard version from failed migration Tue Dec 11 05:50:48 [conn370166] ShardingState::undoDonateChunk acquired _mutex Tue Dec 11 05:50:48 [conn381473] end connection 10.120.13.62:1687 (2739 connections now open) Tue Dec 11 05:50:48 [conn370166] MigrateFromStatus::done About to acquire global write lock to exit critical section Tue Dec 11 05:50:48 [conn370166] MigrateFromStatus::done Global lock acquired Tue Dec 11 05:50:48 [conn370166] SyncClusterConnection connecting to [mdb-l120-13-1:27020] Tue Dec 11 05:50:48 [conn370166] SyncClusterConnection connecting to [mdb-l120-13-2:27020] Tue Dec 11 05:50:48 [conn370166] SyncClusterConnection connecting to [mdb-l120-13-3:27020] Tue Dec 11 05:50:48 [conn370166] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:48 [conn370166] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:48-445", server: "mdbcis4-01-sv", clientAddr: "10.120.13.62:56156", time: new Date(1355201448916), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, step2 of 6: 193, step3 of 6: 65, step4 of 6: 36007447, note: "aborted" } } Tue Dec 11 05:50:48 [conn370166] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 locks(micros) W:27053 r:65339 w:16 reslen:99 36038854ms Tue Dec 11 05:50:50 [conn391596] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:50 [conn391596] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:50 [conn391596] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbaa87f573e75838b2e5 Tue Dec 11 05:50:50 [conn391596] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:50-446", server: "mdbcis4-01-sv", clientAddr: "10.120.13.32:18813", time: new Date(1355201450886), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:50 [conn391596] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:50 [conn391596] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:50 [conn391596] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:50-447", server: "mdbcis4-01-sv", clientAddr: "10.120.13.32:18813", time: new Date(1355201450957), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:50 [conn391596] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 195ms Tue Dec 11 05:50:51 [conn379106] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:51 [conn379106] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:51 [conn379106] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbab87f573e75838b2e6 Tue Dec 11 05:50:51 [conn379106] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:51-448", server: "mdbcis4-01-sv", clientAddr: "10.120.13.42:34588", time: new Date(1355201451952), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:51 [conn379106] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:52 [conn379106] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:52 [conn379106] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:52-449", server: "mdbcis4-01-sv", clientAddr: "10.120.13.42:34588", time: new Date(1355201452057), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:52 [conn379106] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 231ms Tue Dec 11 05:50:52 [conn370170] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:52 [conn370170] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:52 [conn370170] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbac87f573e75838b2e7 Tue Dec 11 05:50:52 [conn370170] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:52-450", server: "mdbcis4-01-sv", clientAddr: "10.120.13.72:52275", time: new Date(1355201452591), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:52 [conn370170] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:52 [conn370170] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:52 [conn370170] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:52-451", server: "mdbcis4-01-sv", clientAddr: "10.120.13.72:52275", time: new Date(1355201452733), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:52 [conn370170] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 338ms Tue Dec 11 05:50:53 [conn392927] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:53 [conn392927] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:53 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbad87f573e75838b2e8 Tue Dec 11 05:50:53 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:53-452", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201453372), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:53 [conn392927] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:53 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:53 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:53-453", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201453514), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:53 [conn392927] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 341ms Tue Dec 11 05:50:54 [conn367583] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:54 [conn367583] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:54 [conn367583] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbae87f573e75838b2e9 Tue Dec 11 05:50:54 [conn367583] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:54-454", server: "mdbcis4-01-sv", clientAddr: "10.120.13.52:25775", time: new Date(1355201454466), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:54 [conn367583] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:54 [conn367583] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:54 [conn367583] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:54-455", server: "mdbcis4-01-sv", clientAddr: "10.120.13.52:25775", time: new Date(1355201454608), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:54 [conn367583] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 338ms Tue Dec 11 05:50:55 [conn370166] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:55 [conn370166] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:55 [conn370166] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbaf87f573e75838b2ea Tue Dec 11 05:50:55 [conn370166] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:55-456", server: "mdbcis4-01-sv", clientAddr: "10.120.13.62:56156", time: new Date(1355201455349), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:55 [conn370166] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:55 [conn370166] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:55 [conn370166] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:55-457", server: "mdbcis4-01-sv", clientAddr: "10.120.13.62:56156", time: new Date(1355201455491), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:55 [conn370166] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 371ms Tue Dec 11 05:50:55 [conn393166] end connection 10.120.13.42:52375 (2738 connections now open) Tue Dec 11 05:50:55 [initandlisten] connection accepted from 10.120.13.42:52383 #393170 (2739 connections now open) Tue Dec 11 05:50:57 [conn391596] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:50:57 [conn391596] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:50:57 [conn391596] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbb187f573e75838b2eb Tue Dec 11 05:50:57 [conn391596] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:57-458", server: "mdbcis4-01-sv", clientAddr: "10.120.13.32:18813", time: new Date(1355201457388), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:50:57 [conn391596] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:50:57 [conn391596] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:50:57 [conn391596] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:50:57-459", server: "mdbcis4-01-sv", clientAddr: "10.120.13.32:18813", time: new Date(1355201457564), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:50:57 [conn391596] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 443ms Tue Dec 11 05:51:03 [initandlisten] connection accepted from 10.121.1.6:36463 #393171 (2740 connections now open) Tue Dec 11 05:51:03 [conn393171] end connection 10.121.1.6:36463 (2739 connections now open) Tue Dec 11 05:51:04 [conn370165] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:04 [conn370165] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:05 [conn370165] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbb887f573e75838b2ec Tue Dec 11 05:51:05 [conn370165] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:05-460", server: "mdbcis4-01-sv", clientAddr: "10.121.13.32:32034", time: new Date(1355201465060), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:05 [conn370165] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:05 [conn370165] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:05 [conn370165] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:05-461", server: "mdbcis4-01-sv", clientAddr: "10.121.13.32:32034", time: new Date(1355201465200), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:05 [conn370165] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 419ms Tue Dec 11 05:51:05 [conn392927] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:05 [conn392927] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:05 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbb987f573e75838b2ed Tue Dec 11 05:51:05 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:05-462", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201465900), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:05 [conn392927] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:05 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:05 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:05-463", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201465971), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:06 [conn392927] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 193ms Tue Dec 11 05:51:13 [conn370167] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:13 [conn370167] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:13 [conn393167] end connection 10.121.13.41:6297 (2738 connections now open) Tue Dec 11 05:51:13 [conn370167] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbc187f573e75838b2ee Tue Dec 11 05:51:13 [conn370167] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:13-464", server: "mdbcis4-01-sv", clientAddr: "10.121.13.52:25094", time: new Date(1355201473346), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:13 [conn370167] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:13 [initandlisten] connection accepted from 10.121.13.41:6301 #393172 (2739 connections now open) Tue Dec 11 05:51:13 [conn370167] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:13 [conn370167] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:13-465", server: "mdbcis4-01-sv", clientAddr: "10.121.13.52:25094", time: new Date(1355201473416), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:13 [conn370167] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 168ms Tue Dec 11 05:51:14 [conn393168] end connection 10.120.13.42:52380 (2738 connections now open) Tue Dec 11 05:51:14 [initandlisten] connection accepted from 10.120.13.42:52388 #393173 (2739 connections now open) Tue Dec 11 05:51:18 [conn393169] end connection 10.121.13.42:20744 (2738 connections now open) Tue Dec 11 05:51:18 [initandlisten] connection accepted from 10.121.13.42:20748 #393174 (2739 connections now open) Tue Dec 11 05:51:22 [conn370161] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:22 [conn370161] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:22 [conn370161] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbca87f573e75838b2ef Tue Dec 11 05:51:22 [conn370161] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:22-466", server: "mdbcis4-01-sv", clientAddr: "10.121.13.72:60726", time: new Date(1355201482461), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:22 [conn370161] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:22 [conn370161] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:22 [conn370161] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:22-467", server: "mdbcis4-01-sv", clientAddr: "10.121.13.72:60726", time: new Date(1355201482533), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:22 [conn370161] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 186ms Tue Dec 11 05:51:23 [conn370170] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:23 [conn370170] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:23 [conn370170] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbcb87f573e75838b2f0 Tue Dec 11 05:51:23 [conn370170] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:23-468", server: "mdbcis4-01-sv", clientAddr: "10.120.13.72:52275", time: new Date(1355201483177), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:23 [conn370170] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:23 [conn370170] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:23 [conn370170] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:23-469", server: "mdbcis4-01-sv", clientAddr: "10.120.13.72:52275", time: new Date(1355201483249), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:23 [conn370170] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 199ms Tue Dec 11 05:51:23 [conn378052] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:23 [conn378052] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:23 [conn378052] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbcb87f573e75838b2f1 Tue Dec 11 05:51:23 [conn378052] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:23-470", server: "mdbcis4-01-sv", clientAddr: "10.120.13.12:46975", time: new Date(1355201483709), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:23 [conn378052] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:23 [conn378052] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:23 [conn378052] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:23-471", server: "mdbcis4-01-sv", clientAddr: "10.120.13.12:46975", time: new Date(1355201483779), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:23 [conn378052] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 188ms Tue Dec 11 05:51:24 [conn392927] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:24 [conn392927] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:24 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbcc87f573e75838b2f2 Tue Dec 11 05:51:24 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:24-472", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201484276), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:24 [conn392927] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:24 [conn392927] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:24 [conn392927] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:24-473", server: "mdbcis4-01-sv", clientAddr: "10.120.13.22:41143", time: new Date(1355201484383), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:24 [conn392927] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 233ms Tue Dec 11 05:51:24 [conn367583] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:24 [conn367583] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:25 [conn367583] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbcc87f573e75838b2f3 Tue Dec 11 05:51:25 [conn367583] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:25-474", server: "mdbcis4-01-sv", clientAddr: "10.120.13.52:25775", time: new Date(1355201485069), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:25 [conn367583] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:25 [conn367583] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked. Tue Dec 11 05:51:25 [conn367583] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:25-475", server: "mdbcis4-01-sv", clientAddr: "10.120.13.52:25775", time: new Date(1355201485196), what: "moveChunk.from", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, step1 of 6: 0, note: "aborted" } } Tue Dec 11 05:51:25 [conn367583] command admin.$cmd command: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } ntoreturn:1 keyUpdates:0 reslen:188 303ms Tue Dec 11 05:51:25 [conn370166] received moveChunk request: { moveChunk: "counters.statistics", from: "shard4/mdbcis4-01-ny.criteo.prod:27021,mdbcis4-01-sv.criteo.prod:27021,mdbcis4-02-ny.criteo.prod:27021,mdbcis4-02-sv.criteo.prod:27021", to: "shard6/mdbcis6-01-ny.criteo.prod:27021,mdbcis6-01-sv.criteo.prod:27021,mdbcis6-02-ny.criteo.prod:27021,mdbcis6-02-sv.criteo.prod:27021", fromShard: "shard4", toShard: "shard6", min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, maxChunkSizeBytes: 67108864, shardId: "counters.statistics-_id_"7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e"", configdb: "mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020", secondaryThrottle: false } Tue Dec 11 05:51:25 [conn370166] created new distributed lock for counters.statistics on mdb-l120-13-1:27020,mdb-l120-13-2:27020,mdb-l120-13-3:27020 ( lock timeout : 900000, ping interval : 30000, process : 0 ) Tue Dec 11 05:51:25 [conn393170] end connection 10.120.13.42:52383 (2738 connections now open) Tue Dec 11 05:51:25 [initandlisten] connection accepted from 10.120.13.42:52391 #393175 (2739 connections now open) Tue Dec 11 05:51:25 [conn370166] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' acquired, ts : 50c6bbcd87f573e75838b2f4 Tue Dec 11 05:51:25 [conn370166] about to log metadata event: { _id: "mdbcis4-01-sv-2012-12-11T04:51:25-476", server: "mdbcis4-01-sv", clientAddr: "10.120.13.62:56156", time: new Date(1355201485974), what: "moveChunk.start", ns: "counters.statistics", details: { min: { _id: "7f8f7d68-5ad5-49e6-92a6-aff7181d6c7e" }, max: { _id: "bfbef656-d8c3-4742-89fc-716a4b3d4c54" }, from: "shard4", to: "shard6" } } Tue Dec 11 05:51:26 [conn370166] warning: aborted moveChunk because official version less than mine?: official 5|1||000000000000000000000000 mine: 6|0||000000000000000000000000 Tue Dec 11 05:51:26 [conn370166] distributed lock 'counters.statistics/mdbcis4-01-sv:27021:1355128470:1131626771' unlocked.