Wed Nov 13 10:24:02 [conn605] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [conn605] Primary for replica set shard6-04 changed to 10.38.168.25:7111 Wed Nov 13 10:24:02 [conn605] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [conn605] Primary for replica set shard6-04 changed to 10.38.168.25:7111 Wed Nov 13 10:24:02 [conn605] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [conn604] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [conn281] Primary for replica set shard6-04 changed to 10.38.168.25:7111 Wed Nov 13 10:24:02 [conn604] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [conn604] Primary for replica set shard6-04 changed to 10.38.168.25:7111 Wed Nov 13 10:24:02 [conn604] DBClientCursor::init call() failed Wed Nov 13 10:24:02 [WriteBackListener-10.57.39.52:7111] new version change detected to 29361|0||522ad499e1814e603d11be30, 23 writebacks processed at 29360|0||522ad499e1814e603d11be30 Wed Nov 13 10:24:02 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:02 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 1 Wed Nov 13 10:24:02 [mongosMain] connection accepted from 10.42.236.43:35397 #3244 (607 connections now open) Wed Nov 13 10:24:02 [conn3244] end connection 10.42.236.43:35397 (606 connections now open) Wed Nov 13 10:24:03 [WriteBackListener-10.57.39.52:7111] ChunkManager: time to load chunks for appid250528.meta_infos0: 215ms sequenceNumber: 310 version: 29361|1||522ad499e1814e603d11be30 based on: 29360|1||522ad499e1814e603d11be30 Wed Nov 13 10:24:03 [conn3243] ChunkManager: time to load chunks for appid250528.meta_infos0: 183ms sequenceNumber: 311 version: 29361|1||522ad499e1814e603d11be30 based on: 29360|1||522ad499e1814e603d11be30 Wed Nov 13 10:24:03 [conn3243] end connection 10.26.17.45:32740 (605 connections now open) Wed Nov 13 10:24:03 [mongosMain] connection accepted from 10.26.17.45:32743 #3245 (606 connections now open) Wed Nov 13 10:24:03 [conn3245] authenticate db: admin { authenticate: 1, user: "structure", nonce: "afd15b6adddc7979", key: "e55f416ae69be9ca5fc74a5ef09a7beb" } Wed Nov 13 10:24:03 [conn3245] CMD: movechunk: { moveChunk: "appid250528.meta_infos0", find: { _uid: 5852291 }, to: "shard6-07", noCleanup: true } Wed Nov 13 10:24:03 [conn3245] moving chunk ns: appid250528.meta_infos0 moving ( ns:appid250528.meta_infos0 at: shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 lastmod: 19689|661||000000000000000000000000 min: { _uid: 5852291 } max: { _uid: 5859919 }) shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 -> shard6-07:shard6-07/10.38.171.25:7111,10.38.171.44:7111,10.38.171.64:7111 Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] new version change detected to 29362|0||522ad499e1814e603d11be30, 15 writebacks processed at 29361|0||522ad499e1814e603d11be30 Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 1 Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 2 Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 3 Wed Nov 13 10:24:20 [WriteBackListener-10.57.39.52:7111] ChunkManager: time to load chunks for appid250528.meta_infos0: 734ms sequenceNumber: 312 version: 29362|1||522ad499e1814e603d11be30 based on: (empty) Wed Nov 13 10:24:21 [conn3245] end connection 10.26.17.45:32743 (605 connections now open) Wed Nov 13 10:24:21 [mongosMain] connection accepted from 10.26.17.45:32750 #3246 (606 connections now open) Wed Nov 13 10:24:21 [conn3246] authenticate db: admin { authenticate: 1, user: "structure", nonce: "157ff559ed698622", key: "c0bb51c80330d1535382663bdb179ffc" } Wed Nov 13 10:24:21 [conn3246] CMD: movechunk: { moveChunk: "appid250528.meta_infos0", find: { _uid: 5912921 }, to: "shard6-07", noCleanup: true } Wed Nov 13 10:24:21 [conn3246] moving chunk ns: appid250528.meta_infos0 moving ( ns:appid250528.meta_infos0 at: shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 lastmod: 1|665||000000000000000000000000 min: { _uid: 5912921 } max: { _uid: 5915905 }) shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 -> shard6-07:shard6-07/10.38.171.25:7111,10.38.171.44:7111,10.38.171.64:7111 Wed Nov 13 10:24:21 [WriteBackListener-10.57.39.52:7111] ChunkManager: time to load chunks for appid250528.meta_infos0: 207ms sequenceNumber: 313 version: 29362|1||522ad499e1814e603d11be30 based on: 29362|1||522ad499e1814e603d11be30 Wed Nov 13 10:24:21 [WriteBackListener-10.57.39.52:7111] warning: chunk manager reload forced for collection 'appid250528.meta_infos0', config version is 29362|1||522ad499e1814e603d11be30 Wed Nov 13 10:24:23 [mongosMain] connection accepted from 10.42.236.43:23556 #3247 (607 connections now open) Wed Nov 13 10:24:23 [conn3247] end connection 10.42.236.43:23556 (606 connections now open) Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] new version change detected, 2 writebacks processed previously Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 1 Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 2 Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] new version change detected, 1 writebacks processed previously Wed Nov 13 10:24:40 [WriteBackListener-10.57.39.52:7111] writeback failed because of stale config, retrying attempts: 3 Wed Nov 13 10:24:41 [WriteBackListener-10.57.39.52:7111] ChunkManager: time to load chunks for appid250528.meta_infos0: 701ms sequenceNumber: 314 version: 29363|1||522ad499e1814e603d11be30 based on: (empty) Wed Nov 13 10:24:41 [conn170] SyncClusterConnection connecting to [10.57.19.58:7222] Wed Nov 13 10:24:41 [conn170] SyncClusterConnection connecting to [10.57.19.55:7222] Wed Nov 13 10:24:41 [conn170] SyncClusterConnection connecting to [10.48.89.35:7222] Wed Nov 13 10:24:41 [conn375] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn170] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn380] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29359000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn301] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn3246] end connection 10.26.17.45:32750 (605 connections now open) Wed Nov 13 10:24:41 [mongosMain] connection accepted from 10.26.17.45:32757 #3248 (606 connections now open) Wed Nov 13 10:24:41 [conn3248] authenticate db: admin { authenticate: 1, user: "structure", nonce: "b2e7a65c4c663184", key: "69cfc1436c36803e28aba3f30a1b0748" } Wed Nov 13 10:24:41 [conn3248] CMD: movechunk: { moveChunk: "appid250528.meta_infos0", find: { _uid: 5974696 }, to: "shard6-07", noCleanup: true } Wed Nov 13 10:24:41 [conn3248] moving chunk ns: appid250528.meta_infos0 moving ( ns:appid250528.meta_infos0 at: shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 lastmod: 14676|22164||000000000000000000000000 min: { _uid: 5974696 } max: { _uid: 5985581 }) shard6-03:shard6-03/10.52.136.60:7111,10.57.39.38:7111,10.57.39.52:7111 -> shard6-07:shard6-07/10.38.171.25:7111,10.38.171.44:7111,10.38.171.64:7111 Wed Nov 13 10:24:41 [conn375] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn170] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn380] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29359000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn301] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn375] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn170] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn380] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29359000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn301] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn375] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [WriteBackListener-10.57.39.52:7111] ChunkManager: time to load chunks for appid250528.meta_infos0: 198ms sequenceNumber: 315 version: 29363|1||522ad499e1814e603d11be30 based on: 29363|1||522ad499e1814e603d11be30 Wed Nov 13 10:24:41 [conn170] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn380] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29359000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn301] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn315] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29361000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn315] going to retry checkShardVersion host: 10.38.171.25:7111 { oldVersion: Timestamp 29361000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn375] setShardVersion failed host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 } Wed Nov 13 10:24:41 [conn375] Assertion: 10429:setShardVersion failed host: 10.38.171.25:7111 { oldVersion: Timestamp 29362000|0, oldVersionEpoch: ObjectId('522ad499e1814e603d11be30'), ns: "appid250528.meta_infos0", version: Timestamp 29362000|0, versionEpoch: ObjectId('522ad499e1814e603d11be30'), globalVersion: Timestamp 29363000|0, globalVersionEpoch: ObjectId('522ad499e1814e603d11be30'), reloadConfig: true, errmsg: "shard global version for collection is higher than trying to set to 'appid250528.meta_infos0'", ok: 0.0 }