2015-08-04T22:34:30.488+0000 I JOURNAL [initandlisten] journal dir=/mnt/tmp/rs2/journal 2015-08-04T22:34:30.488+0000 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed 2015-08-04T22:34:30.490+0000 I JOURNAL [durability] Durability thread started 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] MongoDB starting : pid=12322 port=27018 dbpath=/mnt/tmp/rs2 64-bit host=ip-10-140-200-244 2015-08-04T22:34:30.490+0000 I JOURNAL [journal writer] Journal writer thread started 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] db version v3.0.5 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] git version: 8bc4ae20708dbb493cb09338d9e7be6698e4a3a3 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] build info: Linux ip-10-165-164-140 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 BOOST_LIB_VERSION=1_49 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] allocator: tcmalloc 2015-08-04T22:34:30.490+0000 I CONTROL [initandlisten] options: { net: { port: 27018 }, processManagement: { fork: true }, replication: { replSet: "jon" }, storage: { dbPath: "/mnt/tmp/rs2" }, systemLog: { destination: "file", logAppend: true, path: "/mnt/tmp/rs2.log" } } 2015-08-04T22:34:30.490+0000 I INDEX [initandlisten] allocating new ns file /mnt/tmp/rs2/local.ns, filling with zeroes... 2015-08-04T22:34:30.497+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/local.0, filling with zeroes... 2015-08-04T22:34:30.497+0000 I STORAGE [FileAllocator] creating directory /mnt/tmp/rs2/_tmp 2015-08-04T22:34:30.497+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/local.0, size: 64MB, took 0 secs 2015-08-04T22:34:30.501+0000 I REPL [initandlisten] Did not find local replica set configuration document at startup; NoMatchingDocument Did not find replica set configuration document in local.system.replset 2015-08-04T22:34:30.501+0000 I NETWORK [initandlisten] waiting for connections on port 27018 2015-08-04T22:34:31.351+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42179 #1 (1 connection now open) 2015-08-04T22:34:31.351+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42178 #2 (2 connections now open) 2015-08-04T22:34:31.607+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42181 #3 (3 connections now open) 2015-08-04T22:34:31.608+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42182 #4 (4 connections now open) 2015-08-04T22:34:31.608+0000 I NETWORK [conn1] end connection 10.140.200.244:42179 (2 connections now open) 2015-08-04T22:34:31.608+0000 I NETWORK [conn2] end connection 10.140.200.244:42178 (2 connections now open) 2015-08-04T22:34:31.611+0000 I REPL [WriteReplSetConfig] Starting replication applier threads 2015-08-04T22:34:31.611+0000 I REPL [rsSync] replSet warning did not receive a valid config yet, sleeping 5 seconds 2015-08-04T22:34:31.611+0000 I REPL [ReplicationExecutor] New replica set config in use: { _id: "jon", version: 3, members: [ { _id: 0, host: "ip-10-140-200-244:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "ip-10-140-200-244:27019", arbiterOnly: true, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "ip-10-140-200-244:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatTimeoutSecs: 10, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } } 2015-08-04T22:34:31.611+0000 I REPL [ReplicationExecutor] This node is ip-10-140-200-244:27018 in the config 2015-08-04T22:34:31.611+0000 I REPL [ReplicationExecutor] transition to STARTUP2 2015-08-04T22:34:31.611+0000 I REPL [ReplicationExecutor] Member ip-10-140-200-244:27017 is now in state PRIMARY 2015-08-04T22:34:31.611+0000 I REPL [ReplicationExecutor] Member ip-10-140-200-244:27019 is now in state ARBITER 2015-08-04T22:34:36.611+0000 I REPL [rsSync] ****** 2015-08-04T22:34:36.611+0000 I REPL [rsSync] creating replication oplog of size: 3099MB... 2015-08-04T22:34:36.611+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/local.1, filling with zeroes... 2015-08-04T22:34:36.611+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/local.1, size: 2047MB, took 0 secs 2015-08-04T22:34:36.612+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/local.2, filling with zeroes... 2015-08-04T22:34:36.612+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/local.2, size: 2047MB, took 0 secs 2015-08-04T22:34:36.612+0000 I REPL [rsSync] ****** 2015-08-04T22:34:36.612+0000 I REPL [rsSync] initial sync pending 2015-08-04T22:34:36.612+0000 I REPL [ReplicationExecutor] syncing from: ip-10-140-200-244:27017 2015-08-04T22:34:36.614+0000 I REPL [rsSync] initial sync drop all databases 2015-08-04T22:34:36.614+0000 I STORAGE [rsSync] dropAllDatabasesExceptLocal 1 2015-08-04T22:34:36.614+0000 I REPL [rsSync] initial sync clone all databases 2015-08-04T22:34:36.615+0000 I REPL [rsSync] initial sync cloning db: admin 2015-08-04T22:34:36.616+0000 I REPL [rsSync] initial sync cloning db: POCDB 2015-08-04T22:34:36.616+0000 I INDEX [rsSync] allocating new ns file /mnt/tmp/rs2/POCDB.ns, filling with zeroes... 2015-08-04T22:34:36.623+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.0, filling with zeroes... 2015-08-04T22:34:36.623+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.0, size: 64MB, took 0 secs 2015-08-04T22:34:36.988+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.1, filling with zeroes... 2015-08-04T22:34:36.988+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.1, size: 128MB, took 0 secs 2015-08-04T22:34:37.743+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.2, filling with zeroes... 2015-08-04T22:34:37.744+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.2, size: 256MB, took 0 secs 2015-08-04T22:34:38.792+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.3, filling with zeroes... 2015-08-04T22:34:38.793+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.3, size: 512MB, took 0 secs 2015-08-04T22:34:42.182+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.4, filling with zeroes... 2015-08-04T22:34:42.182+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.4, size: 1024MB, took 0 secs 2015-08-04T22:34:46.799+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.5, filling with zeroes... 2015-08-04T22:34:46.799+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.5, size: 2047MB, took 0 secs 2015-08-04T22:34:53.612+0000 I NETWORK [conn4] end connection 10.140.200.244:42182 (1 connection now open) 2015-08-04T22:34:53.612+0000 I NETWORK [conn3] end connection 10.140.200.244:42181 (1 connection now open) 2015-08-04T22:34:53.613+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42192 #5 (1 connection now open) 2015-08-04T22:34:53.613+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42191 #6 (2 connections now open) 2015-08-04T22:34:55.235+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.6, filling with zeroes... 2015-08-04T22:34:55.235+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.6, size: 2047MB, took 0 secs 2015-08-04T22:35:01.833+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.7, filling with zeroes... 2015-08-04T22:35:01.833+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.7, size: 2047MB, took 0 secs 2015-08-04T22:35:10.676+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.8, filling with zeroes... 2015-08-04T22:35:10.676+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.8, size: 2047MB, took 0 secs 2015-08-04T22:35:22.600+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.9, filling with zeroes... 2015-08-04T22:35:22.600+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.9, size: 2047MB, took 0 secs 2015-08-04T22:35:23.620+0000 I NETWORK [conn6] end connection 10.140.200.244:42191 (1 connection now open) 2015-08-04T22:35:23.620+0000 I NETWORK [conn5] end connection 10.140.200.244:42192 (1 connection now open) 2015-08-04T22:35:23.620+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42197 #7 (1 connection now open) 2015-08-04T22:35:23.620+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42198 #8 (2 connections now open) 2015-08-04T22:35:36.005+0000 I STORAGE [rsSync] clone POCDB.POCCOLL 18061439 2015-08-04T22:35:36.356+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.10, filling with zeroes... 2015-08-04T22:35:36.356+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.10, size: 2047MB, took 0 secs 2015-08-04T22:35:37.014+0000 I STORAGE [rsSync] 18366741 objects cloned so far from collection POCDB.POCCOLL 2015-08-04T22:35:48.276+0000 I JOURNAL [journal writer] old journal file will be removed: /mnt/tmp/rs2/journal/j._0 2015-08-04T22:35:50.177+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.11, filling with zeroes... 2015-08-04T22:35:50.177+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.11, size: 2047MB, took 0 secs 2015-08-04T22:35:53.627+0000 I NETWORK [conn7] end connection 10.140.200.244:42197 (1 connection now open) 2015-08-04T22:35:53.627+0000 I NETWORK [conn8] end connection 10.140.200.244:42198 (1 connection now open) 2015-08-04T22:35:53.628+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42204 #9 (1 connection now open) 2015-08-04T22:35:53.628+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42205 #10 (2 connections now open) 2015-08-04T22:36:04.011+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.12, filling with zeroes... 2015-08-04T22:36:04.011+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.12, size: 2047MB, took 0 secs 2015-08-04T22:36:17.739+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.13, filling with zeroes... 2015-08-04T22:36:17.740+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.13, size: 2047MB, took 0 secs 2015-08-04T22:36:23.634+0000 I NETWORK [conn10] end connection 10.140.200.244:42205 (1 connection now open) 2015-08-04T22:36:23.634+0000 I NETWORK [conn9] end connection 10.140.200.244:42204 (1 connection now open) 2015-08-04T22:36:23.636+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42211 #11 (1 connection now open) 2015-08-04T22:36:23.636+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42210 #12 (2 connections now open) 2015-08-04T22:36:31.159+0000 I INDEX [rsSync] build index on: POCDB.POCCOLL properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "POCDB.POCCOLL" } 2015-08-04T22:36:31.159+0000 I INDEX [rsSync] building index using bulk method 2015-08-04T22:36:38.825+0000 I - [rsSync] Index Build: 2097200/34864640 6% 2015-08-04T22:36:46.471+0000 I - [rsSync] Index Build: 4194400/34864640 12% 2015-08-04T22:36:53.643+0000 I NETWORK [conn12] end connection 10.140.200.244:42210 (1 connection now open) 2015-08-04T22:36:53.643+0000 I NETWORK [conn11] end connection 10.140.200.244:42211 (1 connection now open) 2015-08-04T22:36:53.644+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42217 #13 (1 connection now open) 2015-08-04T22:36:53.644+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42218 #14 (2 connections now open) 2015-08-04T22:36:54.101+0000 I - [rsSync] Index Build: 6291500/34864640 18% 2015-08-04T22:37:01.783+0000 I - [rsSync] Index Build: 8388700/34864640 24% 2015-08-04T22:37:09.449+0000 I - [rsSync] Index Build: 10485800/34864640 30% 2015-08-04T22:37:17.108+0000 I - [rsSync] Index Build: 12583000/34864640 36% 2015-08-04T22:37:23.650+0000 I NETWORK [conn13] end connection 10.140.200.244:42217 (1 connection now open) 2015-08-04T22:37:23.650+0000 I NETWORK [conn14] end connection 10.140.200.244:42218 (1 connection now open) 2015-08-04T22:37:23.650+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42225 #15 (1 connection now open) 2015-08-04T22:37:23.650+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42224 #16 (2 connections now open) 2015-08-04T22:37:24.762+0000 I - [rsSync] Index Build: 14680100/34864640 42% 2015-08-04T22:37:32.430+0000 I - [rsSync] Index Build: 16777300/34864640 48% 2015-08-04T22:37:40.113+0000 I - [rsSync] Index Build: 18874400/34864640 54% 2015-08-04T22:37:47.799+0000 I - [rsSync] Index Build: 20971600/34864640 60% 2015-08-04T22:37:53.657+0000 I NETWORK [conn15] end connection 10.140.200.244:42225 (1 connection now open) 2015-08-04T22:37:53.657+0000 I NETWORK [conn16] end connection 10.140.200.244:42224 (1 connection now open) 2015-08-04T22:37:53.657+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42231 #17 (1 connection now open) 2015-08-04T22:37:53.657+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42232 #18 (2 connections now open) 2015-08-04T22:37:55.499+0000 I - [rsSync] Index Build: 23068700/34864640 66% 2015-08-04T22:38:03.198+0000 I - [rsSync] Index Build: 25165900/34864640 72% 2015-08-04T22:38:10.892+0000 I - [rsSync] Index Build: 27263000/34864640 78% 2015-08-04T22:38:13.005+0000 I - [rsSync] Index Build: 29287400/34864640 84% 2015-08-04T22:38:18.598+0000 I - [rsSync] Index Build: 29360200/34864640 84% 2015-08-04T22:38:23.664+0000 I NETWORK [conn17] end connection 10.140.200.244:42231 (1 connection now open) 2015-08-04T22:38:23.664+0000 I NETWORK [conn18] end connection 10.140.200.244:42232 (1 connection now open) 2015-08-04T22:38:23.664+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42237 #19 (1 connection now open) 2015-08-04T22:38:23.664+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42238 #20 (2 connections now open) 2015-08-04T22:38:26.304+0000 I - [rsSync] Index Build: 31457300/34864640 90% 2015-08-04T22:38:34.009+0000 I - [rsSync] Index Build: 33554500/34864640 96% 2015-08-04T22:38:48.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 7521000/34864640 21% 2015-08-04T22:38:53.671+0000 I NETWORK [conn19] end connection 10.140.200.244:42237 (1 connection now open) 2015-08-04T22:38:53.671+0000 I NETWORK [conn20] end connection 10.140.200.244:42238 (1 connection now open) 2015-08-04T22:38:53.671+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42243 #21 (1 connection now open) 2015-08-04T22:38:53.671+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42244 #22 (2 connections now open) 2015-08-04T22:38:58.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 15641500/34864640 44% 2015-08-04T22:39:08.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 23747000/34864640 68% 2015-08-04T22:39:18.001+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 31879000/34864640 91% 2015-08-04T22:39:21.688+0000 I INDEX [rsSync] done building bottom layer, going to commit 2015-08-04T22:39:21.688+0000 I INDEX [rsSync] build index done. scanned 34864640 total records. 170 secs 2015-08-04T22:39:21.710+0000 I REPL [rsSync] initial sync data copy, starting syncup 2015-08-04T22:39:21.711+0000 I INDEX [rsSync] build index on: POCDB.POCCOLL properties: { v: 1, key: { fld6: 1.0 }, name: "fld6_1", ns: "POCDB.POCCOLL" } 2015-08-04T22:39:21.711+0000 I INDEX [rsSync] building index using bulk method 2015-08-04T22:39:23.678+0000 I NETWORK [conn22] end connection 10.140.200.244:42244 (1 connection now open) 2015-08-04T22:39:23.678+0000 I NETWORK [conn21] end connection 10.140.200.244:42243 (1 connection now open) 2015-08-04T22:39:23.678+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42249 #23 (1 connection now open) 2015-08-04T22:39:23.679+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42250 #24 (2 connections now open) 2015-08-04T22:39:24.001+0000 I - [rsSync] Index Build: 1315000/34864640 3% 2015-08-04T22:39:31.745+0000 I - [rsSync] Index Build: 2688700/34864640 7% 2015-08-04T22:39:34.001+0000 I - [rsSync] Index Build: 3975900/34864640 11% 2015-08-04T22:39:42.012+0000 I - [rsSync] Index Build: 5377400/34864640 15% 2015-08-04T22:39:45.001+0000 I - [rsSync] Index Build: 6949900/34864640 19% 2015-08-04T22:39:52.717+0000 I - [rsSync] Index Build: 8066000/34864640 23% 2015-08-04T22:39:53.684+0000 I NETWORK [conn23] end connection 10.140.200.244:42249 (1 connection now open) 2015-08-04T22:39:53.684+0000 I NETWORK [conn24] end connection 10.140.200.244:42250 (1 connection now open) 2015-08-04T22:39:53.685+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42256 #25 (1 connection now open) 2015-08-04T22:39:53.685+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42257 #26 (2 connections now open) 2015-08-04T22:39:55.001+0000 I - [rsSync] Index Build: 9247100/34864640 26% 2015-08-04T22:40:03.496+0000 I - [rsSync] Index Build: 10754700/34864640 30% 2015-08-04T22:40:06.001+0000 I - [rsSync] Index Build: 12056200/34864640 34% 2015-08-04T22:40:14.204+0000 I - [rsSync] Index Build: 13443300/34864640 38% 2015-08-04T22:40:17.001+0000 I - [rsSync] Index Build: 14987200/34864640 42% 2015-08-04T22:40:23.692+0000 I NETWORK [conn25] end connection 10.140.200.244:42256 (1 connection now open) 2015-08-04T22:40:23.692+0000 I NETWORK [conn26] end connection 10.140.200.244:42257 (1 connection now open) 2015-08-04T22:40:23.692+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42262 #27 (1 connection now open) 2015-08-04T22:40:23.692+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42263 #28 (2 connections now open) 2015-08-04T22:40:24.580+0000 I - [rsSync] Index Build: 16132000/34864640 46% 2015-08-04T22:40:27.001+0000 I - [rsSync] Index Build: 17511300/34864640 50% 2015-08-04T22:40:34.819+0000 I - [rsSync] Index Build: 18820600/34864640 53% 2015-08-04T22:40:37.001+0000 I - [rsSync] Index Build: 20083400/34864640 57% 2015-08-04T22:40:45.009+0000 I - [rsSync] Index Build: 21509300/34864640 61% 2015-08-04T22:40:48.001+0000 I - [rsSync] Index Build: 23248900/34864640 66% 2015-08-04T22:40:53.699+0000 I NETWORK [conn27] end connection 10.140.200.244:42262 (1 connection now open) 2015-08-04T22:40:53.699+0000 I NETWORK [conn28] end connection 10.140.200.244:42263 (1 connection now open) 2015-08-04T22:40:53.699+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42268 #29 (1 connection now open) 2015-08-04T22:40:53.700+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42269 #30 (2 connections now open) 2015-08-04T22:40:55.212+0000 I - [rsSync] Index Build: 24198000/34864640 69% 2015-08-04T22:40:58.001+0000 I - [rsSync] Index Build: 25782300/34864640 73% 2015-08-04T22:41:05.528+0000 I - [rsSync] Index Build: 26886600/34864640 77% 2015-08-04T22:41:08.001+0000 I - [rsSync] Index Build: 28290300/34864640 81% 2015-08-04T22:41:15.794+0000 I - [rsSync] Index Build: 29575300/34864640 84% 2015-08-04T22:41:18.001+0000 I - [rsSync] Index Build: 30818500/34864640 88% 2015-08-04T22:41:23.706+0000 I NETWORK [conn29] end connection 10.140.200.244:42268 (1 connection now open) 2015-08-04T22:41:23.706+0000 I NETWORK [conn30] end connection 10.140.200.244:42269 (1 connection now open) 2015-08-04T22:41:23.706+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42274 #31 (1 connection now open) 2015-08-04T22:41:23.706+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42275 #32 (2 connections now open) 2015-08-04T22:41:26.332+0000 I - [rsSync] Index Build: 32263900/34864640 92% 2015-08-04T22:41:29.001+0000 I - [rsSync] Index Build: 33641900/34864640 96% 2015-08-04T22:41:41.733+0000 I JOURNAL [journal writer] old journal file will be removed: /mnt/tmp/rs2/journal/j._1 2015-08-04T22:41:41.814+0000 I JOURNAL [journal writer] old journal file will be removed: /mnt/tmp/rs2/journal/j._2 2015-08-04T22:41:46.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 9891300/34864640 28% 2015-08-04T22:41:53.712+0000 I NETWORK [conn31] end connection 10.140.200.244:42274 (1 connection now open) 2015-08-04T22:41:53.712+0000 I NETWORK [conn32] end connection 10.140.200.244:42275 (1 connection now open) 2015-08-04T22:41:53.713+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42281 #33 (1 connection now open) 2015-08-04T22:41:53.713+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42280 #34 (2 connections now open) 2015-08-04T22:41:56.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 20566100/34864640 58% 2015-08-04T22:42:06.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 30659800/34864640 87% 2015-08-04T22:42:10.574+0000 I INDEX [rsSync] done building bottom layer, going to commit 2015-08-04T22:42:10.574+0000 I INDEX [rsSync] build index done. scanned 34864640 total records. 168 secs 2015-08-04T22:42:10.596+0000 I REPL [rsSync] oplog sync 1 of 3 2015-08-04T22:42:10.596+0000 I REPL [rsSync] oplog sync 2 of 3 2015-08-04T22:42:10.597+0000 I REPL [rsSync] initial sync building indexes 2015-08-04T22:42:10.597+0000 I REPL [rsSync] initial sync cloning indexes for : admin 2015-08-04T22:42:10.598+0000 I REPL [rsSync] initial sync cloning indexes for : POCDB 2015-08-04T22:42:10.598+0000 I STORAGE [rsSync] copying indexes for: { name: "POCCOLL" } 2015-08-04T22:42:10.599+0000 I INDEX [rsSync] build index on: POCDB.POCCOLL properties: { v: 1, key: { fld0: 1.0 }, name: "fld0_1", ns: "POCDB.POCCOLL" } 2015-08-04T22:42:10.599+0000 I INDEX [rsSync] building index using bulk method 2015-08-04T22:42:10.672+0000 I REPL [ReplicationExecutor] could not find member to sync from 2015-08-04T22:42:13.005+0000 I - [rsSync] Index Build: 1677700/34864640 4% 2015-08-04T22:42:19.956+0000 I - [rsSync] Index Build: 2688700/34864640 7% 2015-08-04T22:42:22.005+0000 I - [rsSync] Index Build: 4040800/34864640 11% 2015-08-04T22:42:23.719+0000 I NETWORK [conn33] end connection 10.140.200.244:42281 (1 connection now open) 2015-08-04T22:42:23.719+0000 I NETWORK [conn34] end connection 10.140.200.244:42280 (1 connection now open) 2015-08-04T22:42:23.720+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42289 #35 (1 connection now open) 2015-08-04T22:42:23.720+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42290 #36 (2 connections now open) 2015-08-04T22:42:29.599+0000 I - [rsSync] Index Build: 5377400/34864640 15% 2015-08-04T22:42:32.005+0000 I - [rsSync] Index Build: 6882800/34864640 19% 2015-08-04T22:42:39.401+0000 I - [rsSync] Index Build: 8066000/34864640 23% 2015-08-04T22:42:42.005+0000 I - [rsSync] Index Build: 9699700/34864640 27% 2015-08-04T22:42:49.211+0000 I - [rsSync] Index Build: 10754700/34864640 30% 2015-08-04T22:42:52.005+0000 I - [rsSync] Index Build: 12516700/34864640 35% 2015-08-04T22:42:53.725+0000 I NETWORK [conn35] end connection 10.140.200.244:42289 (1 connection now open) 2015-08-04T22:42:53.725+0000 I NETWORK [conn36] end connection 10.140.200.244:42290 (1 connection now open) 2015-08-04T22:42:53.726+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42296 #37 (1 connection now open) 2015-08-04T22:42:53.726+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42297 #38 (2 connections now open) 2015-08-04T22:42:59.011+0000 I - [rsSync] Index Build: 13443300/34864640 38% 2015-08-04T22:43:02.005+0000 I - [rsSync] Index Build: 15456800/34864640 44% 2015-08-04T22:43:08.517+0000 I - [rsSync] Index Build: 16132000/34864640 46% 2015-08-04T22:43:11.005+0000 I - [rsSync] Index Build: 17850500/34864640 51% 2015-08-04T22:43:17.950+0000 I - [rsSync] Index Build: 18820600/34864640 53% 2015-08-04T22:43:20.005+0000 I - [rsSync] Index Build: 20272200/34864640 58% 2015-08-04T22:43:23.732+0000 I NETWORK [conn37] end connection 10.140.200.244:42296 (1 connection now open) 2015-08-04T22:43:23.732+0000 I NETWORK [conn38] end connection 10.140.200.244:42297 (1 connection now open) 2015-08-04T22:43:23.733+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42302 #39 (1 connection now open) 2015-08-04T22:43:23.733+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42303 #40 (2 connections now open) 2015-08-04T22:43:27.297+0000 I - [rsSync] Index Build: 21509300/34864640 61% 2015-08-04T22:43:30.005+0000 I - [rsSync] Index Build: 23362600/34864640 67% 2015-08-04T22:43:36.785+0000 I - [rsSync] Index Build: 24198000/34864640 69% 2015-08-04T22:43:39.005+0000 I - [rsSync] Index Build: 25717300/34864640 73% 2015-08-04T22:43:46.261+0000 I - [rsSync] Index Build: 26886600/34864640 77% 2015-08-04T22:43:49.005+0000 I - [rsSync] Index Build: 28766200/34864640 82% 2015-08-04T22:43:53.739+0000 I NETWORK [conn40] end connection 10.140.200.244:42303 (1 connection now open) 2015-08-04T22:43:53.739+0000 I NETWORK [conn39] end connection 10.140.200.244:42302 (1 connection now open) 2015-08-04T22:43:53.740+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42308 #41 (1 connection now open) 2015-08-04T22:43:53.740+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42309 #42 (2 connections now open) 2015-08-04T22:43:55.724+0000 I - [rsSync] Index Build: 29575300/34864640 84% 2015-08-04T22:43:58.005+0000 I - [rsSync] Index Build: 31133900/34864640 89% 2015-08-04T22:44:05.226+0000 I - [rsSync] Index Build: 32263900/34864640 92% 2015-08-04T22:44:08.005+0000 I - [rsSync] Index Build: 34162800/34864640 97% 2015-08-04T22:44:23.746+0000 I NETWORK [conn42] end connection 10.140.200.244:42309 (1 connection now open) 2015-08-04T22:44:23.746+0000 I NETWORK [conn41] end connection 10.140.200.244:42308 (1 connection now open) 2015-08-04T22:44:23.747+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42314 #43 (1 connection now open) 2015-08-04T22:44:23.747+0000 I NETWORK [initandlisten] connection accepted from 10.140.200.244:42315 #44 (2 connections now open) 2015-08-04T22:44:24.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 11071900/34864640 31% 2015-08-04T22:44:30.310+0000 I STORAGE [FileAllocator] allocating new datafile /mnt/tmp/rs2/POCDB.14, filling with zeroes... 2015-08-04T22:44:30.310+0000 I STORAGE [FileAllocator] done allocating datafile /mnt/tmp/rs2/POCDB.14, size: 2047MB, took 0 secs 2015-08-04T22:44:34.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 22403200/34864640 64% 2015-08-04T22:44:44.005+0000 I - [rsSync] Index: (2/3) BTree Bottom Up Progress: 32837700/34864640 94% 2015-08-04T22:44:46.140+0000 I INDEX [rsSync] done building bottom layer, going to commit 2015-08-04T22:44:46.140+0000 I INDEX [rsSync] build index done. scanned 34864640 total records. 155 secs 2015-08-04T22:44:46.164+0000 I REPL [rsSync] oplog sync 3 of 3 2015-08-04T22:44:46.165+0000 I REPL [rsSync] initial sync finishing up 2015-08-04T22:44:46.165+0000 I REPL [rsSync] replSet set minValid=55c13915:1 2015-08-04T22:44:46.165+0000 I REPL [rsSync] initial sync done 2015-08-04T22:44:46.168+0000 I REPL [ReplicationExecutor] transition to RECOVERING 2015-08-04T22:44:46.169+0000 I REPL [ReplicationExecutor] transition to SECONDARY 2015-08-04T22:44:46.184+0000 I JOURNAL [journal writer] old journal file will be removed: /mnt/tmp/rs2/journal/j._3