Fri Dec 14 02:59:15.868 Fri Dec 14 02:59:15.869 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability. Fri Dec 14 02:59:15.869 Fri Dec 14 02:59:15.882 [initandlisten] MongoDB starting : pid=32350 port=27999 dbpath=/data/db/sconsTests/ 32-bit host=bs-rhel-57-32-1 Fri Dec 14 02:59:15.882 [initandlisten] Fri Dec 14 02:59:15.882 [initandlisten] ** NOTE: This is a development version (2.3.2-pre-) of MongoDB. Fri Dec 14 02:59:15.882 [initandlisten] ** Not recommended for production. Fri Dec 14 02:59:15.882 [initandlisten] Fri Dec 14 02:59:15.882 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. Fri Dec 14 02:59:15.882 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal). Fri Dec 14 02:59:15.882 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off. Fri Dec 14 02:59:15.882 [initandlisten] ** See http://www.mongodb.org/display/DOCS/32+bit Fri Dec 14 02:59:15.882 [initandlisten] Fri Dec 14 02:59:15.882 [initandlisten] db version v2.3.2-pre-, pdfile version 4.5 Fri Dec 14 02:59:15.882 [initandlisten] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 Fri Dec 14 02:59:15.882 [initandlisten] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 Fri Dec 14 02:59:15.882 [initandlisten] options: { dbpath: "/data/db/sconsTests/", port: 27999, setParameter: [ "enableTestCommands=1" ] } Fri Dec 14 02:59:15.886 [initandlisten] Unable to check for journal files due to: boost::filesystem::directory_iterator::construct: No such file or directory: "/data/db/sconsTests/journal" Fri Dec 14 02:59:15.933 [FileAllocator] allocating new datafile /data/db/sconsTests/local.ns, filling with zeroes... Fri Dec 14 02:59:15.933 [FileAllocator] creating directory /data/db/sconsTests/_tmp Fri Dec 14 02:59:15.978 [FileAllocator] done allocating datafile /data/db/sconsTests/local.ns, size: 16MB, took 0.037 secs Fri Dec 14 02:59:15.979 [FileAllocator] allocating new datafile /data/db/sconsTests/local.0, filling with zeroes... Fri Dec 14 02:59:16.017 [FileAllocator] done allocating datafile /data/db/sconsTests/local.0, size: 16MB, took 0.037 secs Fri Dec 14 02:59:16.021 [initandlisten] waiting for connections on port 27999 Fri Dec 14 02:59:16.021 [websvr] admin web console waiting for connections on port 28999 Fri Dec 14 02:59:16.828 [initandlisten] connection accepted from 127.0.0.1:53821 #1 (1 connection now open) Fri Dec 14 02:59:16.828 [conn1] end connection 127.0.0.1:53821 (0 connections now open) MongoDB shell version: 2.3.2-pre- null Resetting db path '/data/db/sync20' Fri Dec 14 02:59:17.430 shell: started program /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/mongod --port 30000 --dbpath /data/db/sync20 --setParameter enableTestCommands=1 m30000| Fri Dec 14 02:59:17.466 m30000| Fri Dec 14 02:59:17.466 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability. m30000| Fri Dec 14 02:59:17.466 m30000| Fri Dec 14 02:59:17.477 [initandlisten] MongoDB starting : pid=32365 port=30000 dbpath=/data/db/sync20 32-bit host=bs-rhel-57-32-1 m30000| Fri Dec 14 02:59:17.477 [initandlisten] m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** NOTE: This is a development version (2.3.2-pre-) of MongoDB. m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** Not recommended for production. m30000| Fri Dec 14 02:59:17.477 [initandlisten] m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal). m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off. m30000| Fri Dec 14 02:59:17.477 [initandlisten] ** See http://www.mongodb.org/display/DOCS/32+bit m30000| Fri Dec 14 02:59:17.477 [initandlisten] m30000| Fri Dec 14 02:59:17.477 [initandlisten] db version v2.3.2-pre-, pdfile version 4.5 m30000| Fri Dec 14 02:59:17.477 [initandlisten] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 m30000| Fri Dec 14 02:59:17.478 [initandlisten] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 m30000| Fri Dec 14 02:59:17.478 [initandlisten] options: { dbpath: "/data/db/sync20", port: 30000, setParameter: [ "enableTestCommands=1" ] } m30000| Fri Dec 14 02:59:17.478 [initandlisten] Unable to check for journal files due to: boost::filesystem::directory_iterator::construct: No such file or directory: "/data/db/sync20/journal" m30000| Fri Dec 14 02:59:17.508 [FileAllocator] allocating new datafile /data/db/sync20/local.ns, filling with zeroes... m30000| Fri Dec 14 02:59:17.508 [FileAllocator] creating directory /data/db/sync20/_tmp m30000| Fri Dec 14 02:59:17.552 [FileAllocator] done allocating datafile /data/db/sync20/local.ns, size: 16MB, took 0.036 secs m30000| Fri Dec 14 02:59:17.552 [FileAllocator] allocating new datafile /data/db/sync20/local.0, filling with zeroes... m30000| Fri Dec 14 02:59:17.589 [FileAllocator] done allocating datafile /data/db/sync20/local.0, size: 16MB, took 0.036 secs m30000| Fri Dec 14 02:59:17.591 [initandlisten] waiting for connections on port 30000 m30000| Fri Dec 14 02:59:17.591 [websvr] admin web console waiting for connections on port 31000 m30000| Fri Dec 14 02:59:17.632 [initandlisten] connection accepted from 127.0.0.1:48258 #1 (1 connection now open) Resetting db path '/data/db/sync21' Fri Dec 14 02:59:17.638 shell: started program /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/mongod --port 30001 --dbpath /data/db/sync21 --setParameter enableTestCommands=1 m30001| Fri Dec 14 02:59:17.675 m30001| Fri Dec 14 02:59:17.675 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability. m30001| Fri Dec 14 02:59:17.675 m30001| Fri Dec 14 02:59:17.686 [initandlisten] MongoDB starting : pid=32380 port=30001 dbpath=/data/db/sync21 32-bit host=bs-rhel-57-32-1 m30001| Fri Dec 14 02:59:17.686 [initandlisten] m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** NOTE: This is a development version (2.3.2-pre-) of MongoDB. m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** Not recommended for production. m30001| Fri Dec 14 02:59:17.686 [initandlisten] m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal). m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off. m30001| Fri Dec 14 02:59:17.686 [initandlisten] ** See http://www.mongodb.org/display/DOCS/32+bit m30001| Fri Dec 14 02:59:17.686 [initandlisten] m30001| Fri Dec 14 02:59:17.686 [initandlisten] db version v2.3.2-pre-, pdfile version 4.5 m30001| Fri Dec 14 02:59:17.686 [initandlisten] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 m30001| Fri Dec 14 02:59:17.686 [initandlisten] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 m30001| Fri Dec 14 02:59:17.686 [initandlisten] options: { dbpath: "/data/db/sync21", port: 30001, setParameter: [ "enableTestCommands=1" ] } m30001| Fri Dec 14 02:59:17.686 [initandlisten] Unable to check for journal files due to: boost::filesystem::directory_iterator::construct: No such file or directory: "/data/db/sync21/journal" m30001| Fri Dec 14 02:59:17.696 [FileAllocator] allocating new datafile /data/db/sync21/local.ns, filling with zeroes... m30001| Fri Dec 14 02:59:17.696 [FileAllocator] creating directory /data/db/sync21/_tmp m30001| Fri Dec 14 02:59:17.740 [FileAllocator] done allocating datafile /data/db/sync21/local.ns, size: 16MB, took 0.037 secs m30001| Fri Dec 14 02:59:17.741 [FileAllocator] allocating new datafile /data/db/sync21/local.0, filling with zeroes... m30001| Fri Dec 14 02:59:17.779 [FileAllocator] done allocating datafile /data/db/sync21/local.0, size: 16MB, took 0.037 secs m30001| Fri Dec 14 02:59:17.781 [websvr] admin web console waiting for connections on port 31001 m30001| Fri Dec 14 02:59:17.781 [initandlisten] waiting for connections on port 30001 m30001| Fri Dec 14 02:59:17.840 [initandlisten] connection accepted from 127.0.0.1:38193 #1 (1 connection now open) Resetting db path '/data/db/sync22' Fri Dec 14 02:59:17.852 shell: started program /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/mongod --port 30002 --dbpath /data/db/sync22 --setParameter enableTestCommands=1 m30002| Fri Dec 14 02:59:17.881 m30002| Fri Dec 14 02:59:17.881 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability. m30002| Fri Dec 14 02:59:17.881 m30002| Fri Dec 14 02:59:17.892 [initandlisten] MongoDB starting : pid=32395 port=30002 dbpath=/data/db/sync22 32-bit host=bs-rhel-57-32-1 m30002| Fri Dec 14 02:59:17.892 [initandlisten] m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** NOTE: This is a development version (2.3.2-pre-) of MongoDB. m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** Not recommended for production. m30002| Fri Dec 14 02:59:17.892 [initandlisten] m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal). m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off. m30002| Fri Dec 14 02:59:17.892 [initandlisten] ** See http://www.mongodb.org/display/DOCS/32+bit m30002| Fri Dec 14 02:59:17.892 [initandlisten] m30002| Fri Dec 14 02:59:17.892 [initandlisten] db version v2.3.2-pre-, pdfile version 4.5 m30002| Fri Dec 14 02:59:17.892 [initandlisten] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 m30002| Fri Dec 14 02:59:17.892 [initandlisten] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 m30002| Fri Dec 14 02:59:17.892 [initandlisten] options: { dbpath: "/data/db/sync22", port: 30002, setParameter: [ "enableTestCommands=1" ] } m30002| Fri Dec 14 02:59:17.892 [initandlisten] Unable to check for journal files due to: boost::filesystem::directory_iterator::construct: No such file or directory: "/data/db/sync22/journal" m30002| Fri Dec 14 02:59:17.904 [FileAllocator] allocating new datafile /data/db/sync22/local.ns, filling with zeroes... m30002| Fri Dec 14 02:59:17.904 [FileAllocator] creating directory /data/db/sync22/_tmp m30002| Fri Dec 14 02:59:17.949 [FileAllocator] done allocating datafile /data/db/sync22/local.ns, size: 16MB, took 0.038 secs m30002| Fri Dec 14 02:59:17.949 [FileAllocator] allocating new datafile /data/db/sync22/local.0, filling with zeroes... m30002| Fri Dec 14 02:59:17.989 [FileAllocator] done allocating datafile /data/db/sync22/local.0, size: 16MB, took 0.039 secs m30002| Fri Dec 14 02:59:17.991 [initandlisten] waiting for connections on port 30002 m30002| Fri Dec 14 02:59:17.991 [websvr] admin web console waiting for connections on port 31002 "localhost:30000,localhost:30001,localhost:30002" Fri Dec 14 02:59:18.057 SyncClusterConnection connecting to [localhost:30000] m30000| Fri Dec 14 02:59:18.059 [initandlisten] connection accepted from 127.0.0.1:48263 #2 (2 connections now open) Fri Dec 14 02:59:18.060 SyncClusterConnection connecting to [localhost:30001] m30002| Fri Dec 14 02:59:18.060 [initandlisten] connection accepted from 127.0.0.1:40301 #1 (1 connection now open) m30001| Fri Dec 14 02:59:18.060 [initandlisten] connection accepted from 127.0.0.1:38197 #2 (2 connections now open) Fri Dec 14 02:59:18.061 SyncClusterConnection connecting to [localhost:30002] m30002| Fri Dec 14 02:59:18.062 [initandlisten] connection accepted from 127.0.0.1:40304 #2 (2 connections now open) ShardingTest sync2 : { "config" : "localhost:30000,localhost:30001,localhost:30002", "shards" : [ connection to localhost:30000, connection to localhost:30001, connection to localhost:30002 ] } Fri Dec 14 02:59:18.066 shell: started program /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/mongos --port 30999 --configdb localhost:30000,localhost:30001,localhost:30002 -vvvvvvvvvv --chunkSize 50 --setParameter enableTestCommands=1 m30999| Fri Dec 14 02:59:18.507 [mongosMain] MongoS version 2.3.2-pre- starting: pid=32416 port=30999 32-bit host=bs-rhel-57-32-1 (--help for usage) m30999| Fri Dec 14 02:59:18.507 [mongosMain] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 m30999| Fri Dec 14 02:59:18.507 [mongosMain] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 m30999| Fri Dec 14 02:59:18.507 [mongosMain] options: { chunkSize: 50, configdb: "localhost:30000,localhost:30001,localhost:30002", port: 30999, setParameter: [ "enableTestCommands=1" ], vvvvvvvvvv: true } m30999| Fri Dec 14 02:59:18.509 [mongosMain] config string : localhost:30000,localhost:30001,localhost:30002 m30999| Fri Dec 14 02:59:18.509 [mongosMain] creating new connection to:localhost:30000 m30999| Fri Dec 14 02:59:18.510 BackgroundJob starting: ConnectBG m30000| Fri Dec 14 02:59:18.510 [initandlisten] connection accepted from 127.0.0.1:48269 #3 (3 connections now open) m30999| Fri Dec 14 02:59:18.510 [mongosMain] connected connection! m30999| Fri Dec 14 02:59:18.512 [mongosMain] creating new connection to:localhost:30001 m30999| Fri Dec 14 02:59:18.512 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:18.512 [mongosMain] connected connection! m30001| Fri Dec 14 02:59:18.513 [initandlisten] connection accepted from 127.0.0.1:38203 #3 (3 connections now open) m30999| Fri Dec 14 02:59:18.514 [mongosMain] creating new connection to:localhost:30002 m30999| Fri Dec 14 02:59:18.514 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:18.514 [mongosMain] connected connection! m30002| Fri Dec 14 02:59:18.514 [initandlisten] connection accepted from 127.0.0.1:40310 #3 (3 connections now open) m30999| Fri Dec 14 02:59:18.515 [mongosMain] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:18.515 BackgroundJob starting: CheckConfigServers m30999| Fri Dec 14 02:59:18.515 BackgroundJob starting: ConnectBG m30000| Fri Dec 14 02:59:18.515 [initandlisten] connection accepted from 127.0.0.1:48272 #4 (4 connections now open) m30999| Fri Dec 14 02:59:18.515 [mongosMain] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:18.515 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:18.515 [initandlisten] connection accepted from 127.0.0.1:38206 #4 (4 connections now open) m30999| Fri Dec 14 02:59:18.516 [mongosMain] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:18.516 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:18.516 [initandlisten] connection accepted from 127.0.0.1:40313 #4 (4 connections now open) m30000| Fri Dec 14 02:59:18.518 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:19.249 [conn4] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:1 reslen:51 731ms m30001| Fri Dec 14 02:59:19.250 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:19.940 [conn4] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:2 reslen:51 690ms m30002| Fri Dec 14 02:59:19.940 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:20.676 [conn4] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:2 reslen:51 735ms m30000| Fri Dec 14 02:59:20.676 [FileAllocator] allocating new datafile /data/db/sync20/config.ns, filling with zeroes... m30001| Fri Dec 14 02:59:20.676 [FileAllocator] allocating new datafile /data/db/sync21/config.ns, filling with zeroes... m30002| Fri Dec 14 02:59:20.722 [FileAllocator] allocating new datafile /data/db/sync22/config.ns, filling with zeroes... m30002| Fri Dec 14 02:59:20.766 [FileAllocator] done allocating datafile /data/db/sync22/config.ns, size: 16MB, took 0.043 secs m30001| Fri Dec 14 02:59:20.766 [FileAllocator] done allocating datafile /data/db/sync21/config.ns, size: 16MB, took 0.089 secs m30000| Fri Dec 14 02:59:20.766 [FileAllocator] done allocating datafile /data/db/sync20/config.ns, size: 16MB, took 0.089 secs m30002| Fri Dec 14 02:59:20.766 [FileAllocator] allocating new datafile /data/db/sync22/config.0, filling with zeroes... m30001| Fri Dec 14 02:59:20.766 [FileAllocator] allocating new datafile /data/db/sync21/config.0, filling with zeroes... m30000| Fri Dec 14 02:59:20.782 [FileAllocator] allocating new datafile /data/db/sync20/config.0, filling with zeroes... m30002| Fri Dec 14 02:59:20.814 [FileAllocator] done allocating datafile /data/db/sync22/config.0, size: 16MB, took 0.046 secs m30002| Fri Dec 14 02:59:20.816 [conn4] build index config.version { _id: 1 } m30002| Fri Dec 14 02:59:20.816 [conn4] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:20.816 [conn4] insert config.version keyUpdates:0 locks(micros) w:140524 140ms m30002| Fri Dec 14 02:59:20.817 [FileAllocator] allocating new datafile /data/db/sync22/config.1, filling with zeroes... m30001| Fri Dec 14 02:59:20.902 [FileAllocator] done allocating datafile /data/db/sync21/config.0, size: 16MB, took 0.135 secs m30000| Fri Dec 14 02:59:20.902 [FileAllocator] done allocating datafile /data/db/sync20/config.0, size: 16MB, took 0.119 secs m30000| Fri Dec 14 02:59:20.904 [conn4] build index config.version { _id: 1 } m30000| Fri Dec 14 02:59:20.905 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:20.905 [conn4] insert config.version keyUpdates:0 locks(micros) w:229300 229ms m30000| Fri Dec 14 02:59:20.905 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:20.939 [FileAllocator] done allocating datafile /data/db/sync22/config.1, size: 32MB, took 0.121 secs m30001| Fri Dec 14 02:59:20.904 [conn4] build index config.version { _id: 1 } m30001| Fri Dec 14 02:59:20.905 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:20.905 [conn4] insert config.version keyUpdates:0 locks(micros) w:228872 228ms m30001| Fri Dec 14 02:59:20.905 [FileAllocator] allocating new datafile /data/db/sync21/config.1, filling with zeroes... m30000| Fri Dec 14 02:59:20.908 [FileAllocator] allocating new datafile /data/db/sync20/config.1, filling with zeroes... m30000| Fri Dec 14 02:59:21.416 [FileAllocator] done allocating datafile /data/db/sync20/config.1, size: 32MB, took 0.507 secs m30001| Fri Dec 14 02:59:21.416 [FileAllocator] done allocating datafile /data/db/sync21/config.1, size: 32MB, took 0.511 secs m30000| Fri Dec 14 02:59:21.940 [conn4] command admin.$cmd command: { getlasterror: 1, fsync: 1 } ntoreturn:1 keyUpdates:0 reslen:83 1034ms m30001| Fri Dec 14 02:59:21.940 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:22.798 [conn4] command admin.$cmd command: { getlasterror: 1, fsync: 1 } ntoreturn:1 keyUpdates:0 reslen:83 858ms m30002| Fri Dec 14 02:59:22.799 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.467 [conn4] command admin.$cmd command: { getlasterror: 1, fsync: 1 } ntoreturn:1 keyUpdates:0 reslen:83 668ms m30999| Fri Dec 14 02:59:23.467 [mongosMain] Sending command { ismaster: 1 } to localhost:30000 with $auth: {} m30999| Fri Dec 14 02:59:23.467 [mongosMain] Sending command { ismaster: 1 } to localhost:30001 with $auth: {} m30999| Fri Dec 14 02:59:23.467 [mongosMain] Sending command { ismaster: 1 } to localhost:30002 with $auth: {} m30000| Fri Dec 14 02:59:23.470 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.470 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.471 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.471 [conn4] build index config.settings { _id: 1 } m30000| Fri Dec 14 02:59:23.472 [conn4] build index done. scanned 0 total records. 0.001 secs m30000| Fri Dec 14 02:59:23.473 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.472 [conn4] build index config.settings { _id: 1 } m30001| Fri Dec 14 02:59:23.478 [conn4] build index done. scanned 0 total records. 0.005 secs m30002| Fri Dec 14 02:59:23.477 [conn4] build index config.settings { _id: 1 } m30002| Fri Dec 14 02:59:23.478 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.480 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.488 [conn4] fsync from getlasterror m30000| Fri Dec 14 02:59:23.493 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.493 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.493 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.494 [conn4] build index config.chunks { _id: 1 } m30001| Fri Dec 14 02:59:23.494 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.494 [conn4] info: creating collection config.chunks on add index m30001| Fri Dec 14 02:59:23.494 [conn4] build index config.chunks { ns: 1, min: 1 } m30001| Fri Dec 14 02:59:23.495 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.494 [conn4] build index config.chunks { _id: 1 } m30000| Fri Dec 14 02:59:23.494 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.494 [conn4] info: creating collection config.chunks on add index m30000| Fri Dec 14 02:59:23.494 [conn4] build index config.chunks { ns: 1, min: 1 } m30000| Fri Dec 14 02:59:23.495 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.495 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.502 [conn4] build index config.chunks { _id: 1 } m30002| Fri Dec 14 02:59:23.503 [conn4] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:23.503 [conn4] info: creating collection config.chunks on add index m30002| Fri Dec 14 02:59:23.503 [conn4] build index config.chunks { ns: 1, min: 1 } m30002| Fri Dec 14 02:59:23.504 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.505 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.511 [conn4] fsync from getlasterror m30000| Fri Dec 14 02:59:23.520 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.520 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.520 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.521 [conn4] build index config.chunks { ns: 1, shard: 1, min: 1 } m30000| Fri Dec 14 02:59:23.521 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.521 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.521 [conn4] build index config.chunks { ns: 1, shard: 1, min: 1 } m30002| Fri Dec 14 02:59:23.521 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.521 [conn4] build index config.chunks { ns: 1, shard: 1, min: 1 } m30001| Fri Dec 14 02:59:23.525 [conn4] build index done. scanned 0 total records. 0.004 secs m30001| Fri Dec 14 02:59:23.528 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.534 [conn4] fsync from getlasterror m30000| Fri Dec 14 02:59:23.539 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.539 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.539 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.540 [conn4] build index config.chunks { ns: 1, lastmod: 1 } m30000| Fri Dec 14 02:59:23.540 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.540 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.544 [conn4] build index config.chunks { ns: 1, lastmod: 1 } m30002| Fri Dec 14 02:59:23.545 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.545 [conn4] build index config.chunks { ns: 1, lastmod: 1 } m30001| Fri Dec 14 02:59:23.545 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.547 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.554 [conn4] fsync from getlasterror m30000| Fri Dec 14 02:59:23.559 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.559 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.559 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.559 [conn4] build index config.shards { _id: 1 } m30000| Fri Dec 14 02:59:23.560 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.560 [conn4] info: creating collection config.shards on add index m30000| Fri Dec 14 02:59:23.560 [conn4] build index config.shards { host: 1 } m30000| Fri Dec 14 02:59:23.561 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.561 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.566 [conn4] build index config.shards { _id: 1 } m30002| Fri Dec 14 02:59:23.567 [conn4] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:23.567 [conn4] info: creating collection config.shards on add index m30002| Fri Dec 14 02:59:23.567 [conn4] build index config.shards { host: 1 } m30002| Fri Dec 14 02:59:23.567 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.567 [conn4] build index config.shards { _id: 1 } m30001| Fri Dec 14 02:59:23.568 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.568 [conn4] info: creating collection config.shards on add index m30001| Fri Dec 14 02:59:23.568 [conn4] build index config.shards { host: 1 } m30001| Fri Dec 14 02:59:23.569 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.571 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.579 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.587 [mongosMain] fd limit hard:1024 soft:1024 max conn: 819 m30999| Fri Dec 14 02:59:23.587 BackgroundJob starting: cursorTimeout m30999| Fri Dec 14 02:59:23.587 BackgroundJob starting: Balancer m30999| Fri Dec 14 02:59:23.587 BackgroundJob starting: PeriodicTask::Runner m30999| Fri Dec 14 02:59:23.588 [websvr] fd limit hard:1024 soft:1024 max conn: 819 m30999| Fri Dec 14 02:59:23.588 [websvr] admin web console waiting for connections on port 31999 m30999| Fri Dec 14 02:59:23.588 [Balancer] about to contact config servers and shards m30999| Fri Dec 14 02:59:23.588 [Balancer] config servers and shards contacted successfully m30999| Fri Dec 14 02:59:23.588 [Balancer] balancer id: bs-rhel-57-32-1:30999 started at Dec 14 02:59:23 m30001| Fri Dec 14 02:59:23.590 [conn4] build index config.mongos { _id: 1 } m30001| Fri Dec 14 02:59:23.591 [conn4] build index done. scanned 0 total records. 0 secs m30999| Fri Dec 14 02:59:23.588 [Balancer] created new distributed lock for balancer on localhost:30000,localhost:30001,localhost:30002 ( lock timeout : 900000, ping interval : 30000, process : 0 ) m30000| Fri Dec 14 02:59:23.589 [conn4] build index config.mongos { _id: 1 } m30000| Fri Dec 14 02:59:23.589 [conn4] build index done. scanned 0 total records. 0 secs m30999| Fri Dec 14 02:59:23.591 [mongosMain] waiting for connections on port 30999 m30002| Fri Dec 14 02:59:23.591 [conn4] build index config.mongos { _id: 1 } m30002| Fri Dec 14 02:59:23.592 [conn4] build index done. scanned 0 total records. 0 secs m30999| Fri Dec 14 02:59:23.592 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:23.592 [mongosMain] connection accepted from 127.0.0.1:60949 #1 (1 connection now open) m30999| Fri Dec 14 02:59:23.593 BackgroundJob starting: ConnectBG m30000| Fri Dec 14 02:59:23.597 [initandlisten] connection accepted from 127.0.0.1:48304 #5 (5 connections now open) Fri Dec 14 02:59:23.597 shell: started program /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/mongos --port 30998 --configdb localhost:30000,localhost:30001,localhost:30002 -vvvvvvvvvv --chunkSize 50 --setParameter enableTestCommands=1 m30999| Fri Dec 14 02:59:23.603 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:23.603 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:23.603 [initandlisten] connection accepted from 127.0.0.1:38239 #5 (5 connections now open) m30999| Fri Dec 14 02:59:23.603 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:23.603 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.603 [initandlisten] connection accepted from 127.0.0.1:40346 #5 (5 connections now open) m30999| Fri Dec 14 02:59:23.609 [Balancer] Refreshing MaxChunkSize: 50 m30998| Fri Dec 14 02:59:23.609 [mongosMain] MongoS version 2.3.2-pre- starting: pid=32465 port=30998 32-bit host=bs-rhel-57-32-1 (--help for usage) m30998| Fri Dec 14 02:59:23.609 [mongosMain] git version: c9837ea549a76089bb1122d8ec06a4fc2c634077 m30998| Fri Dec 14 02:59:23.609 [mongosMain] build info: Linux bs-rhel-57-32-1 2.6.18-274.el5xen #1 SMP Fri Jul 8 18:18:40 EDT 2011 i686 BOOST_LIB_VERSION=1_49 m30998| Fri Dec 14 02:59:23.609 [mongosMain] options: { chunkSize: 50, configdb: "localhost:30000,localhost:30001,localhost:30002", port: 30998, setParameter: [ "enableTestCommands=1" ], vvvvvvvvvv: true } m30998| Fri Dec 14 02:59:23.609 [mongosMain] config string : localhost:30000,localhost:30001,localhost:30002 m30000| Fri Dec 14 02:59:23.612 [initandlisten] connection accepted from 127.0.0.1:48308 #6 (6 connections now open) m30998| Fri Dec 14 02:59:23.609 [mongosMain] creating new connection to:localhost:30000 m30998| Fri Dec 14 02:59:23.609 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.609 [mongosMain] connected connection! m30999| Fri Dec 14 02:59:23.614 [Balancer] creating new connection to:localhost:30000 m30999| Fri Dec 14 02:59:23.614 BackgroundJob starting: ConnectBG m30000| Fri Dec 14 02:59:23.614 [initandlisten] connection accepted from 127.0.0.1:48309 #7 (7 connections now open) m30999| Fri Dec 14 02:59:23.615 [Balancer] connected connection! m30999| Fri Dec 14 02:59:23.615 [Balancer] skew from remote server localhost:30000 found: 0 m30999| Fri Dec 14 02:59:23.615 [Balancer] creating new connection to:localhost:30001 m30999| Fri Dec 14 02:59:23.615 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.615 [Balancer] connected connection! m30001| Fri Dec 14 02:59:23.617 [initandlisten] connection accepted from 127.0.0.1:38243 #6 (6 connections now open) m30998| Fri Dec 14 02:59:23.617 [mongosMain] creating new connection to:localhost:30001 m30998| Fri Dec 14 02:59:23.617 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:23.618 [initandlisten] connection accepted from 127.0.0.1:38245 #7 (7 connections now open) m30998| Fri Dec 14 02:59:23.618 [mongosMain] connected connection! m30999| Fri Dec 14 02:59:23.617 [Balancer] skew from remote server localhost:30001 found: -1 m30999| Fri Dec 14 02:59:23.617 [Balancer] creating new connection to:localhost:30002 m30999| Fri Dec 14 02:59:23.617 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.617 [Balancer] connected connection! m30002| Fri Dec 14 02:59:23.620 [initandlisten] connection accepted from 127.0.0.1:40350 #6 (6 connections now open) m30998| Fri Dec 14 02:59:23.622 [mongosMain] creating new connection to:localhost:30002 m30998| Fri Dec 14 02:59:23.622 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.622 [mongosMain] connected connection! m30002| Fri Dec 14 02:59:23.622 [initandlisten] connection accepted from 127.0.0.1:40352 #7 (7 connections now open) m30999| Fri Dec 14 02:59:23.625 [Balancer] skew from remote server localhost:30002 found: -6 m30999| Fri Dec 14 02:59:23.625 [Balancer] skew from remote server localhost:30000 found: 0 m30999| Fri Dec 14 02:59:23.625 [Balancer] skew from remote server localhost:30001 found: 0 m30999| Fri Dec 14 02:59:23.626 [Balancer] skew from remote server localhost:30002 found: 0 m30999| Fri Dec 14 02:59:23.626 [Balancer] skew from remote server localhost:30000 found: 0 m30999| Fri Dec 14 02:59:23.627 [Balancer] skew from remote server localhost:30001 found: -1 m30998| Fri Dec 14 02:59:23.627 [mongosMain] SyncClusterConnection connecting to [localhost:30000] m30998| Fri Dec 14 02:59:23.627 BackgroundJob starting: CheckConfigServers m30998| Fri Dec 14 02:59:23.627 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.627 [mongosMain] SyncClusterConnection connecting to [localhost:30001] m30998| Fri Dec 14 02:59:23.627 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:23.627 [initandlisten] connection accepted from 127.0.0.1:38248 #8 (8 connections now open) m30998| Fri Dec 14 02:59:23.627 [mongosMain] SyncClusterConnection connecting to [localhost:30002] m30998| Fri Dec 14 02:59:23.627 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.629 [initandlisten] connection accepted from 127.0.0.1:40355 #8 (8 connections now open) m30000| Fri Dec 14 02:59:23.629 [initandlisten] connection accepted from 127.0.0.1:48314 #8 (8 connections now open) m30998| Fri Dec 14 02:59:23.629 [mongosMain] MaxChunkSize: 50 m30000| Fri Dec 14 02:59:23.629 [conn8] CMD fsync: sync:1 lock:0 m30999| Fri Dec 14 02:59:23.633 [Balancer] skew from remote server localhost:30002 found: 0 m30999| Fri Dec 14 02:59:23.633 [Balancer] total clock skew of 2ms for servers localhost:30000,localhost:30001,localhost:30002 is in 30000ms bounds. m30999| Fri Dec 14 02:59:23.633 [LockPinger] creating distributed lock ping thread for localhost:30000,localhost:30001,localhost:30002 and process bs-rhel-57-32-1:30999:1355471963:1804289383 (sleeping for 30000ms) m30999| Fri Dec 14 02:59:23.633 [LockPinger] distributed lock pinger 'localhost:30000,localhost:30001,localhost:30002/bs-rhel-57-32-1:30999:1355471963:1804289383' about to ping. m30000| Fri Dec 14 02:59:23.634 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.637 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.637 [conn8] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.642 [conn5] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.642 [conn8] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.648 [conn5] build index config.lockpings { _id: 1 } m30000| Fri Dec 14 02:59:23.648 [conn5] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.649 [conn5] build index config.lockpings { _id: 1 } m30000| Fri Dec 14 02:59:23.649 [conn8] fsync from getlasterror m30000| Fri Dec 14 02:59:23.650 [conn5] fsync from getlasterror m30001| Fri Dec 14 02:59:23.653 [conn5] build index done. scanned 0 total records. 0.004 secs m30002| Fri Dec 14 02:59:23.654 [conn5] build index config.lockpings { _id: 1 } m30002| Fri Dec 14 02:59:23.654 [conn5] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.657 [conn8] fsync from getlasterror m30001| Fri Dec 14 02:59:23.657 [conn5] fsync from getlasterror m30999| Fri Dec 14 02:59:23.659 [Balancer] trying to acquire new distributed lock for balancer on localhost:30000,localhost:30001,localhost:30002 ( lock timeout : 900000, ping interval : 30000, process : bs-rhel-57-32-1:30999:1355471963:1804289383 ) m30000| Fri Dec 14 02:59:23.659 [initandlisten] connection accepted from 127.0.0.1:48317 #9 (9 connections now open) m30002| Fri Dec 14 02:59:23.660 [initandlisten] connection accepted from 127.0.0.1:40358 #9 (9 connections now open) m30000| Fri Dec 14 02:59:23.660 [conn9] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.661 [initandlisten] connection accepted from 127.0.0.1:38251 #9 (9 connections now open) m30001| Fri Dec 14 02:59:23.661 [conn9] CMD fsync: sync:1 lock:0 m30999| Fri Dec 14 02:59:23.659 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:23.659 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.659 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:23.660 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.660 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:23.660 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.660 [Balancer] inserting initial doc in config.locks for lock balancer m30002| Fri Dec 14 02:59:23.663 [conn5] fsync from getlasterror m30002| Fri Dec 14 02:59:23.663 [conn9] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.666 [conn8] fsync from getlasterror m30000| Fri Dec 14 02:59:23.668 [conn8] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.668 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.668 [conn8] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.668 [conn8] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.668 [conn8] fsync from getlasterror m30001| Fri Dec 14 02:59:23.668 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.669 [conn8] fsync from getlasterror m30002| Fri Dec 14 02:59:23.669 [conn8] fsync from getlasterror m30000| Fri Dec 14 02:59:23.669 [conn8] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.669 [conn9] build index config.locks { _id: 1 } m30000| Fri Dec 14 02:59:23.670 [conn9] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.670 [conn9] fsync from getlasterror m30001| Fri Dec 14 02:59:23.670 [conn9] build index config.locks { _id: 1 } m30001| Fri Dec 14 02:59:23.670 [conn9] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:23.674 [conn9] build index config.locks { _id: 1 } m30002| Fri Dec 14 02:59:23.674 [conn9] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:23.677 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.678 [conn8] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.679 [conn8] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.679 [conn9] fsync from getlasterror m30002| Fri Dec 14 02:59:23.679 [conn9] fsync from getlasterror m30000| Fri Dec 14 02:59:23.684 [conn8] fsync from getlasterror m30001| Fri Dec 14 02:59:23.684 [conn8] fsync from getlasterror m30002| Fri Dec 14 02:59:23.684 [conn8] fsync from getlasterror m30000| Fri Dec 14 02:59:23.684 [conn8] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.684 [conn8] CMD fsync: sync:1 lock:0 m30999| Fri Dec 14 02:59:23.685 [Balancer] about to acquire distributed lock 'balancer/bs-rhel-57-32-1:30999:1355471963:1804289383: m30999| { "state" : 1, m30999| "who" : "bs-rhel-57-32-1:30999:1355471963:1804289383:Balancer:846930886", m30999| "process" : "bs-rhel-57-32-1:30999:1355471963:1804289383", m30999| "when" : { "$date" : "Fri Dec 14 02:59:23 2012" }, m30999| "why" : "doing balance round", m30999| "ts" : { "$oid" : "50cadc5b8387ff1af5ba3bb6" } } m30999| { "_id" : "balancer", m30999| "state" : 0 } m30000| Fri Dec 14 02:59:23.685 [conn9] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.685 [conn9] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.686 [conn8] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.686 [conn9] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.688 [conn8] fsync from getlasterror m30000| Fri Dec 14 02:59:23.688 [conn5] fsync from getlasterror m30001| Fri Dec 14 02:59:23.688 [conn5] fsync from getlasterror m30002| Fri Dec 14 02:59:23.689 [conn5] fsync from getlasterror m30000| Fri Dec 14 02:59:23.689 [conn5] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.689 [conn8] fsync from getlasterror m30002| Fri Dec 14 02:59:23.689 [conn8] fsync from getlasterror m30998| Fri Dec 14 02:59:23.691 [mongosMain] fd limit hard:1024 soft:1024 max conn: 819 m30998| Fri Dec 14 02:59:23.691 BackgroundJob starting: cursorTimeout m30998| Fri Dec 14 02:59:23.691 [mongosMain] waiting for connections on port 30998 m30000| Fri Dec 14 02:59:23.692 [conn9] fsync from getlasterror m30998| Fri Dec 14 02:59:23.691 BackgroundJob starting: Balancer m30998| Fri Dec 14 02:59:23.691 [Balancer] about to contact config servers and shards m30998| Fri Dec 14 02:59:23.691 [websvr] fd limit hard:1024 soft:1024 max conn: 819 m30998| Fri Dec 14 02:59:23.691 [websvr] admin web console waiting for connections on port 31998 m30998| Fri Dec 14 02:59:23.691 BackgroundJob starting: PeriodicTask::Runner m30998| Fri Dec 14 02:59:23.691 [Balancer] config servers and shards contacted successfully m30998| Fri Dec 14 02:59:23.692 [Balancer] balancer id: bs-rhel-57-32-1:30998 started at Dec 14 02:59:23 m30998| Fri Dec 14 02:59:23.692 [Balancer] created new distributed lock for balancer on localhost:30000,localhost:30001,localhost:30002 ( lock timeout : 900000, ping interval : 30000, process : 0 ) m30998| Fri Dec 14 02:59:23.692 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30000| Fri Dec 14 02:59:23.692 [initandlisten] connection accepted from 127.0.0.1:48321 #10 (10 connections now open) m30998| Fri Dec 14 02:59:23.692 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.693 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30001| Fri Dec 14 02:59:23.693 [conn5] CMD fsync: sync:1 lock:0 m30998| Fri Dec 14 02:59:23.693 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:23.693 [initandlisten] connection accepted from 127.0.0.1:38255 #10 (10 connections now open) m30998| Fri Dec 14 02:59:23.693 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30998| Fri Dec 14 02:59:23.693 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.693 [initandlisten] connection accepted from 127.0.0.1:40362 #10 (10 connections now open) m30998| Fri Dec 14 02:59:23.693 [Balancer] Refreshing MaxChunkSize: 50 m30998| Fri Dec 14 02:59:23.694 [Balancer] creating new connection to:localhost:30000 m30998| Fri Dec 14 02:59:23.694 BackgroundJob starting: ConnectBG m30000| Fri Dec 14 02:59:23.694 [initandlisten] connection accepted from 127.0.0.1:48324 #11 (11 connections now open) m30998| Fri Dec 14 02:59:23.694 [Balancer] connected connection! m30998| Fri Dec 14 02:59:23.695 [Balancer] skew from remote server localhost:30000 found: -1 m30001| Fri Dec 14 02:59:23.695 [conn9] fsync from getlasterror m30998| Fri Dec 14 02:59:23.695 [Balancer] creating new connection to:localhost:30001 m30998| Fri Dec 14 02:59:23.695 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.695 [Balancer] connected connection! m30001| Fri Dec 14 02:59:23.695 [initandlisten] connection accepted from 127.0.0.1:38258 #11 (11 connections now open) m30998| Fri Dec 14 02:59:23.695 [Balancer] skew from remote server localhost:30001 found: 0 m30998| Fri Dec 14 02:59:23.695 [Balancer] creating new connection to:localhost:30002 m30998| Fri Dec 14 02:59:23.696 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:23.696 [Balancer] connected connection! m30002| Fri Dec 14 02:59:23.696 [initandlisten] connection accepted from 127.0.0.1:40365 #11 (11 connections now open) m30998| Fri Dec 14 02:59:23.696 [Balancer] skew from remote server localhost:30002 found: 0 m30998| Fri Dec 14 02:59:23.697 [Balancer] skew from remote server localhost:30000 found: -1 m30998| Fri Dec 14 02:59:23.697 [Balancer] skew from remote server localhost:30001 found: 0 m30998| Fri Dec 14 02:59:23.698 [Balancer] skew from remote server localhost:30002 found: -1 m30002| Fri Dec 14 02:59:23.698 [conn9] fsync from getlasterror m30002| Fri Dec 14 02:59:23.698 [conn5] CMD fsync: sync:1 lock:0 m30998| Fri Dec 14 02:59:23.698 [Balancer] skew from remote server localhost:30000 found: 0 m30998| Fri Dec 14 02:59:23.699 [Balancer] skew from remote server localhost:30001 found: 0 m30998| Fri Dec 14 02:59:23.699 [Balancer] skew from remote server localhost:30002 found: 0 m30998| Fri Dec 14 02:59:23.699 [Balancer] total clock skew of 0ms for servers localhost:30000,localhost:30001,localhost:30002 is in 30000ms bounds. m30998| Fri Dec 14 02:59:23.699 [Balancer] trying to acquire new distributed lock for balancer on localhost:30000,localhost:30001,localhost:30002 ( lock timeout : 900000, ping interval : 30000, process : bs-rhel-57-32-1:30998:1355471963:1804289383 ) m30998| Fri Dec 14 02:59:23.699 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30998| Fri Dec 14 02:59:23.700 [LockPinger] creating distributed lock ping thread for localhost:30000,localhost:30001,localhost:30002 and process bs-rhel-57-32-1:30998:1355471963:1804289383 (sleeping for 30000ms) m30998| Fri Dec 14 02:59:23.700 [LockPinger] distributed lock pinger 'localhost:30000,localhost:30001,localhost:30002/bs-rhel-57-32-1:30998:1355471963:1804289383' about to ping. m30000| Fri Dec 14 02:59:23.700 [conn10] CMD fsync: sync:1 lock:0 m30998| Fri Dec 14 02:59:23.700 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:23.700 [conn10] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.700 [initandlisten] connection accepted from 127.0.0.1:48327 #12 (12 connections now open) m30998| Fri Dec 14 02:59:23.700 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30998| Fri Dec 14 02:59:23.701 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.701 [conn10] CMD fsync: sync:1 lock:0 m30998| Fri Dec 14 02:59:23.701 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30001| Fri Dec 14 02:59:23.701 [initandlisten] connection accepted from 127.0.0.1:38261 #12 (12 connections now open) m30998| Fri Dec 14 02:59:23.701 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.701 [initandlisten] connection accepted from 127.0.0.1:40368 #12 (12 connections now open) m30998| Fri Dec 14 02:59:23.701 [Balancer] checking last ping for lock 'balancer/bs-rhel-57-32-1:30999:1355471963:1804289383' against process and ping Wed Dec 31 19:00:00 1969 m30998| Fri Dec 14 02:59:23.702 [Balancer] could not force lock 'balancer/bs-rhel-57-32-1:30999:1355471963:1804289383' because elapsed time 0 <= takeover time 900000 m30998| Fri Dec 14 02:59:23.702 [Balancer] skipping balancing round because another balancer is active m30000| Fri Dec 14 02:59:23.702 [conn9] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.703 [conn5] build index config.lockpings { ping: new Date(1) } m30000| Fri Dec 14 02:59:23.704 [conn5] build index done. scanned 1 total records. 0 secs m30000| Fri Dec 14 02:59:23.704 [conn5] fsync from getlasterror m30002| Fri Dec 14 02:59:23.703 [conn5] build index config.lockpings { ping: new Date(1) } m30002| Fri Dec 14 02:59:23.704 [conn5] build index done. scanned 1 total records. 0 secs m30001| Fri Dec 14 02:59:23.704 [conn5] build index config.lockpings { ping: new Date(1) } m30000| Fri Dec 14 02:59:23.704 [conn10] fsync from getlasterror m30001| Fri Dec 14 02:59:23.705 [conn5] build index done. scanned 1 total records. 0 secs m30001| Fri Dec 14 02:59:23.714 [conn10] fsync from getlasterror m30001| Fri Dec 14 02:59:23.713 [conn9] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.714 [conn5] fsync from getlasterror m30002| Fri Dec 14 02:59:23.719 [conn9] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.719 [conn5] fsync from getlasterror m30002| Fri Dec 14 02:59:23.719 [conn10] fsync from getlasterror m30000| Fri Dec 14 02:59:23.725 [conn9] fsync from getlasterror m30999| Fri Dec 14 02:59:23.725 [LockPinger] cluster localhost:30000,localhost:30001,localhost:30002 pinged successfully at Fri Dec 14 02:59:23 2012 by distributed lock pinger 'localhost:30000,localhost:30001,localhost:30002/bs-rhel-57-32-1:30999:1355471963:1804289383', sleeping for 30000ms m30000| Fri Dec 14 02:59:23.726 [conn10] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.727 [conn9] fsync from getlasterror m30001| Fri Dec 14 02:59:23.728 [conn10] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.729 [conn9] fsync from getlasterror m30002| Fri Dec 14 02:59:23.729 [conn10] CMD fsync: sync:1 lock:0 m30999| Fri Dec 14 02:59:23.730 [Balancer] distributed lock 'balancer/bs-rhel-57-32-1:30999:1355471963:1804289383' acquired, ts : 50cadc5b8387ff1af5ba3bb6 m30999| Fri Dec 14 02:59:23.730 [Balancer] *** start balancing round m30999| Fri Dec 14 02:59:23.730 [Balancer] no collections to balance m30999| Fri Dec 14 02:59:23.730 [Balancer] no need to move any chunk m30000| Fri Dec 14 02:59:23.730 [conn10] fsync from getlasterror m30999| Fri Dec 14 02:59:23.730 [Balancer] *** end of balancing round m30001| Fri Dec 14 02:59:23.730 [conn10] fsync from getlasterror m30000| Fri Dec 14 02:59:23.730 [conn9] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.730 [conn10] fsync from getlasterror m30001| Fri Dec 14 02:59:23.730 [conn9] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.731 [conn10] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.731 [conn9] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.731 [conn10] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.731 [conn10] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.731 [conn10] fsync from getlasterror m30000| Fri Dec 14 02:59:23.731 [conn9] fsync from getlasterror m30001| Fri Dec 14 02:59:23.733 [conn9] fsync from getlasterror m30001| Fri Dec 14 02:59:23.733 [conn10] fsync from getlasterror m30002| Fri Dec 14 02:59:23.735 [conn9] fsync from getlasterror m30002| Fri Dec 14 02:59:23.735 [conn10] fsync from getlasterror m30999| Fri Dec 14 02:59:23.737 [Balancer] distributed lock 'balancer/bs-rhel-57-32-1:30999:1355471963:1804289383' unlocked. m30998| Fri Dec 14 02:59:23.737 [LockPinger] cluster localhost:30000,localhost:30001,localhost:30002 pinged successfully at Fri Dec 14 02:59:23 2012 by distributed lock pinger 'localhost:30000,localhost:30001,localhost:30002/bs-rhel-57-32-1:30998:1355471963:1804289383', sleeping for 30000ms m30998| Fri Dec 14 02:59:23.801 [mongosMain] connection accepted from 127.0.0.1:46073 #1 (1 connection now open) ShardingTest undefined going to add shard : localhost:30000 m30999| Fri Dec 14 02:59:23.803 [conn1] couldn't find database [admin] in config db m30000| Fri Dec 14 02:59:23.803 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.806 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.807 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.809 [conn4] build index config.databases { _id: 1 } m30000| Fri Dec 14 02:59:23.809 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:23.809 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.809 [conn4] build index config.databases { _id: 1 } m30001| Fri Dec 14 02:59:23.809 [conn4] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:23.813 [conn4] build index config.databases { _id: 1 } m30002| Fri Dec 14 02:59:23.813 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:23.816 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.820 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.826 [conn1] put [admin] on: config:localhost:30000,localhost:30001,localhost:30002 m30999| Fri Dec 14 02:59:23.826 [conn1] Request::process begin ns: admin.$cmd msg id: 0 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.826 [conn1] single query: admin.$cmd { addshard: "localhost:30000" } ntoreturn: -1 options : 0 m30999| Fri Dec 14 02:59:23.828 [conn1] going to add shard: { _id: "shard0000", host: "localhost:30000" } m30000| Fri Dec 14 02:59:23.828 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.828 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.828 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.829 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.833 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.837 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.840 [conn1] Request::process end ns: admin.$cmd msg id: 0 op: 2004 attempt: 0 14ms { "shardAdded" : "shard0000", "ok" : 1 } ShardingTest undefined going to add shard : localhost:30001 m30999| Fri Dec 14 02:59:23.842 [conn1] Request::process begin ns: admin.$cmd msg id: 1 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.842 [conn1] single query: admin.$cmd { addshard: "localhost:30001" } ntoreturn: -1 options : 0 m30999| Fri Dec 14 02:59:23.843 [conn1] going to add shard: { _id: "shard0001", host: "localhost:30001" } m30000| Fri Dec 14 02:59:23.844 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.844 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.844 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.844 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.849 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.853 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.856 [conn1] Request::process end ns: admin.$cmd msg id: 1 op: 2004 attempt: 0 13ms { "shardAdded" : "shard0001", "ok" : 1 } ShardingTest undefined going to add shard : localhost:30002 m30999| Fri Dec 14 02:59:23.856 [conn1] Request::process begin ns: admin.$cmd msg id: 2 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.856 [conn1] single query: admin.$cmd { addshard: "localhost:30002" } ntoreturn: -1 options : 0 m30999| Fri Dec 14 02:59:23.857 [conn1] going to add shard: { _id: "shard0002", host: "localhost:30002" } m30000| Fri Dec 14 02:59:23.857 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.858 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.858 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.858 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.863 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.867 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.875 [conn1] Request::process end ns: admin.$cmd msg id: 2 op: 2004 attempt: 0 18ms { "shardAdded" : "shard0002", "ok" : 1 } m30999| Fri Dec 14 02:59:23.876 [conn1] Request::process begin ns: config.settings msg id: 3 op: 2001 attempt: 0 m30999| Fri Dec 14 02:59:23.876 [conn1] write: config.settings m30999| Fri Dec 14 02:59:23.880 [conn1] creating new connection to:localhost:30000 m30999| Fri Dec 14 02:59:23.880 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.880 [conn1] connected connection! m30999| Fri Dec 14 02:59:23.880 [conn1] creating WriteBackListener for: localhost:30000 serverID: 50cadc5b8387ff1af5ba3bb5 m30999| Fri Dec 14 02:59:23.880 [conn1] initializing shard connection to localhost:30000 m30999| Fri Dec 14 02:59:23.880 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } m30999| Fri Dec 14 02:59:23.880 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } to localhost:30000 with $auth: {} m30000| Fri Dec 14 02:59:23.880 [initandlisten] connection accepted from 127.0.0.1:48333 #13 (13 connections now open) m30999| Fri Dec 14 02:59:23.880 BackgroundJob starting: WriteBackListener-localhost:30000 m30999| Fri Dec 14 02:59:23.881 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30999| Fri Dec 14 02:59:23.881 [conn1] creating new connection to:localhost:30001 m30999| Fri Dec 14 02:59:23.881 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.881 [conn1] connected connection! m30999| Fri Dec 14 02:59:23.881 [conn1] creating WriteBackListener for: localhost:30001 serverID: 50cadc5b8387ff1af5ba3bb5 m30999| Fri Dec 14 02:59:23.881 [conn1] initializing shard connection to localhost:30001 m30001| Fri Dec 14 02:59:23.881 [initandlisten] connection accepted from 127.0.0.1:38267 #13 (13 connections now open) m30999| Fri Dec 14 02:59:23.881 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } m30999| Fri Dec 14 02:59:23.881 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } to localhost:30001 with $auth: {} m30999| Fri Dec 14 02:59:23.881 BackgroundJob starting: WriteBackListener-localhost:30001 m30999| Fri Dec 14 02:59:23.881 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30999| Fri Dec 14 02:59:23.881 [conn1] creating new connection to:localhost:30002 m30999| Fri Dec 14 02:59:23.881 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.881 [conn1] connected connection! m30999| Fri Dec 14 02:59:23.881 [conn1] creating WriteBackListener for: localhost:30002 serverID: 50cadc5b8387ff1af5ba3bb5 m30999| Fri Dec 14 02:59:23.881 [conn1] initializing shard connection to localhost:30002 m30999| Fri Dec 14 02:59:23.881 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } m30999| Fri Dec 14 02:59:23.881 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true } to localhost:30002 with $auth: {} m30002| Fri Dec 14 02:59:23.881 [initandlisten] connection accepted from 127.0.0.1:40374 #13 (13 connections now open) m30999| Fri Dec 14 02:59:23.882 BackgroundJob starting: WriteBackListener-localhost:30002 m30999| Fri Dec 14 02:59:23.882 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30999| Fri Dec 14 02:59:23.882 [conn1] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:23.882 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.882 [conn1] SyncClusterConnection connecting to [localhost:30001] m30000| Fri Dec 14 02:59:23.882 [initandlisten] connection accepted from 127.0.0.1:48336 #14 (14 connections now open) m30999| Fri Dec 14 02:59:23.882 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.882 [conn1] SyncClusterConnection connecting to [localhost:30002] m30001| Fri Dec 14 02:59:23.882 [initandlisten] connection accepted from 127.0.0.1:38270 #14 (14 connections now open) m30999| Fri Dec 14 02:59:23.882 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.882 [initandlisten] connection accepted from 127.0.0.1:40377 #14 (14 connections now open) m30000| Fri Dec 14 02:59:23.882 [conn14] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.883 [conn14] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.883 [conn14] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.883 [conn14] fsync from getlasterror m30001| Fri Dec 14 02:59:23.887 [conn14] fsync from getlasterror m30002| Fri Dec 14 02:59:23.891 [conn14] fsync from getlasterror m30999| Fri Dec 14 02:59:23.893 [conn1] Request::process end ns: config.settings msg id: 3 op: 2001 attempt: 0 17ms m30999| Fri Dec 14 02:59:23.893 [conn1] Request::process begin ns: config.mongos msg id: 4 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.893 [conn1] shard query: config.mongos {} m30999| Fri Dec 14 02:59:23.899 [conn1] creating pcursor over QSpec { ns: "config.mongos", n2skip: 0, n2return: 0, options: 0, query: {}, fields: {} } and CInfo { v_ns: "", filter: {} } m30999| Fri Dec 14 02:59:23.899 [conn1] initializing over 1 shards required by [unsharded @ config:localhost:30000,localhost:30001,localhost:30002] m30999| Fri Dec 14 02:59:23.899 [conn1] initializing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30999| Fri Dec 14 02:59:23.899 [conn1] initialized query (full) on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "bs-rhel-57-32-1:30999", ping: new Date(1355471963737), up: 0, waiting: true }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.899 [conn1] finishing over 1 shards m30999| Fri Dec 14 02:59:23.899 [conn1] finishing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "bs-rhel-57-32-1:30999", ping: new Date(1355471963737), up: 0, waiting: true }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.899 [conn1] finished on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "(done)", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "bs-rhel-57-32-1:30999", ping: new Date(1355471963737), up: 0, waiting: true }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.899 [conn1] Request::process end ns: config.mongos msg id: 4 op: 2004 attempt: 0 5ms Waiting for active hosts... Waiting for the balancer lock... m30999| Fri Dec 14 02:59:23.900 [conn1] Request::process begin ns: config.locks msg id: 5 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.900 [conn1] shard query: config.locks { _id: "balancer" } m30999| Fri Dec 14 02:59:23.900 [conn1] creating pcursor over QSpec { ns: "config.locks", n2skip: 0, n2return: -1, options: 0, query: { _id: "balancer" }, fields: {} } and CInfo { v_ns: "", filter: {} } m30999| Fri Dec 14 02:59:23.900 [conn1] initializing over 1 shards required by [unsharded @ config:localhost:30000,localhost:30001,localhost:30002] m30999| Fri Dec 14 02:59:23.900 [conn1] initializing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30999| Fri Dec 14 02:59:23.900 [conn1] initialized query (full) on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "balancer", process: "bs-rhel-57-32-1:30999:1355471963:1804289383", state: 0, ts: ObjectId('50cadc5b8387ff1af5ba3bb6'), when: new Date(1355471963685), who: "bs-rhel-57-32-1:30999:1355471963:1804289383:Balancer:846930886", why: "doing balance round" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.900 [conn1] finishing over 1 shards m30999| Fri Dec 14 02:59:23.900 [conn1] finishing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "balancer", process: "bs-rhel-57-32-1:30999:1355471963:1804289383", state: 0, ts: ObjectId('50cadc5b8387ff1af5ba3bb6'), when: new Date(1355471963685), who: "bs-rhel-57-32-1:30999:1355471963:1804289383:Balancer:846930886", why: "doing balance round" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.900 [conn1] finished on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "(done)", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "balancer", process: "bs-rhel-57-32-1:30999:1355471963:1804289383", state: 0, ts: ObjectId('50cadc5b8387ff1af5ba3bb6'), when: new Date(1355471963685), who: "bs-rhel-57-32-1:30999:1355471963:1804289383:Balancer:846930886", why: "doing balance round" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:23.900 [conn1] Request::process end ns: config.locks msg id: 5 op: 2004 attempt: 0 0ms Waiting again for active hosts after balancer is off... m30999| Fri Dec 14 02:59:23.901 [conn1] Request::process begin ns: admin.$cmd msg id: 6 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.901 [conn1] single query: admin.$cmd { enablesharding: "test" } ntoreturn: -1 options : 0 m30999| Fri Dec 14 02:59:23.901 [conn1] couldn't find database [test] in config db m30999| Fri Dec 14 02:59:23.901 [conn1] creating new connection to:localhost:30000 m30999| Fri Dec 14 02:59:23.901 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.901 [conn1] connected connection! m30000| Fri Dec 14 02:59:23.901 [initandlisten] connection accepted from 127.0.0.1:48339 #15 (15 connections now open) m30999| Fri Dec 14 02:59:23.902 [conn1] creating new connection to:localhost:30001 m30999| Fri Dec 14 02:59:23.902 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:23.902 [conn1] connected connection! m30001| Fri Dec 14 02:59:23.902 [initandlisten] connection accepted from 127.0.0.1:38273 #15 (15 connections now open) m30999| Fri Dec 14 02:59:23.903 [conn1] creating new connection to:localhost:30002 m30999| Fri Dec 14 02:59:23.903 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:23.903 [initandlisten] connection accepted from 127.0.0.1:40380 #15 (15 connections now open) m30999| Fri Dec 14 02:59:23.903 [conn1] connected connection! m30999| Fri Dec 14 02:59:23.903 [conn1] best shard for new allocation is shard: shard0000:localhost:30000 mapped: 64 writeLock: 0 m30000| Fri Dec 14 02:59:23.904 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.904 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.904 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.904 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.908 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.912 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.914 [conn1] put [test] on: shard0000:localhost:30000 m30999| Fri Dec 14 02:59:23.914 [conn1] enabling sharding on: test m30000| Fri Dec 14 02:59:23.914 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:23.914 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:23.915 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:23.915 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:23.917 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:23.919 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:23.920 [conn1] Request::process end ns: admin.$cmd msg id: 6 op: 2004 attempt: 0 19ms m30999| Fri Dec 14 02:59:23.921 [conn1] Request::process begin ns: admin.$cmd msg id: 7 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:23.921 [conn1] single query: admin.$cmd { shardcollection: "test.foo", key: { num: 1.0 } } ntoreturn: -1 options : 0 m30000| Fri Dec 14 02:59:23.923 [FileAllocator] allocating new datafile /data/db/sync20/test.ns, filling with zeroes... m30000| Fri Dec 14 02:59:23.968 [FileAllocator] done allocating datafile /data/db/sync20/test.ns, size: 16MB, took 0.044 secs m30000| Fri Dec 14 02:59:23.968 [FileAllocator] allocating new datafile /data/db/sync20/test.0, filling with zeroes... m30000| Fri Dec 14 02:59:24.009 [FileAllocator] done allocating datafile /data/db/sync20/test.0, size: 16MB, took 0.04 secs m30000| Fri Dec 14 02:59:24.011 [conn15] build index test.foo { _id: 1 } m30000| Fri Dec 14 02:59:24.012 [conn15] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:24.012 [conn15] info: creating collection test.foo on add index m30000| Fri Dec 14 02:59:24.012 [conn15] build index test.foo { num: 1.0 } m30000| Fri Dec 14 02:59:24.013 [conn15] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:24.013 [FileAllocator] allocating new datafile /data/db/sync20/test.1, filling with zeroes... m30999| Fri Dec 14 02:59:24.039 [conn1] CMD: shardcollection: { shardcollection: "test.foo", key: { num: 1.0 } } m30999| Fri Dec 14 02:59:24.039 [conn1] enable sharding on: test.foo with shard key: { num: 1.0 } m30999| Fri Dec 14 02:59:24.040 [conn1] going to create 1 chunk(s) for: test.foo using new epoch 50cadc5c8387ff1af5ba3bb7 m30000| Fri Dec 14 02:59:24.043 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.445 [FileAllocator] done allocating datafile /data/db/sync20/test.1, size: 32MB, took 0.432 secs m30000| Fri Dec 14 02:59:24.795 [conn4] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:2 reslen:51 751ms m30001| Fri Dec 14 02:59:24.795 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.795 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.796 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:24.802 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:24.807 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:24.812 [conn1] major version query from 0|0||50cadc5c8387ff1af5ba3bb7 and over 0 shards is { ns: "test.foo", $or: [ { lastmod: { $gte: Timestamp 0|0 } } ] } m30999| Fri Dec 14 02:59:24.812 [conn1] found 1 new chunks for collection test.foo (tracking 1), new version is 0xab26f60 m30999| Fri Dec 14 02:59:24.812 [conn1] loaded 1 chunks into new chunk manager for test.foo with version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.812 [conn1] ChunkManager: time to load chunks for test.foo: 0ms sequenceNumber: 2 version: 1|0||50cadc5c8387ff1af5ba3bb7 based on: (empty) m30000| Fri Dec 14 02:59:24.813 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.813 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.813 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.813 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:24.815 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:24.818 [conn4] fsync from getlasterror m30000| Fri Dec 14 02:59:24.820 [conn4] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.820 [conn4] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.820 [conn4] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.820 [conn4] build index config.collections { _id: 1 } m30000| Fri Dec 14 02:59:24.821 [conn4] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:24.821 [conn4] fsync from getlasterror m30001| Fri Dec 14 02:59:24.825 [conn4] build index config.collections { _id: 1 } m30001| Fri Dec 14 02:59:24.825 [conn4] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:24.824 [conn4] build index config.collections { _id: 1 } m30002| Fri Dec 14 02:59:24.825 [conn4] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:24.827 [conn4] fsync from getlasterror m30002| Fri Dec 14 02:59:24.834 [conn4] fsync from getlasterror m30999| Fri Dec 14 02:59:24.839 [conn1] have to set shard version for conn: localhost:30000 ns:test.foo my last seq: 0 current: 2 version: 1|0||50cadc5c8387ff1af5ba3bb7 manager: 0xab26ec0 m30999| Fri Dec 14 02:59:24.839 [conn1] setShardVersion shard0000 localhost:30000 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0000", shardHost: "localhost:30000" } 0xab202c8 2 m30999| Fri Dec 14 02:59:24.839 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0000", shardHost: "localhost:30000" } to localhost:30000 with $auth: {} m30999| Fri Dec 14 02:59:24.839 [conn1] setShardVersion failed! m30999| { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ns: "test.foo", need_authoritative: true, errmsg: "first time for collection 'test.foo'", ok: 0.0 } m30999| Fri Dec 14 02:59:24.842 [conn1] have to set shard version for conn: localhost:30000 ns:test.foo my last seq: 0 current: 2 version: 1|0||50cadc5c8387ff1af5ba3bb7 manager: 0xab26ec0 m30999| Fri Dec 14 02:59:24.842 [conn1] setShardVersion shard0000 localhost:30000 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true, shard: "shard0000", shardHost: "localhost:30000" } 0xab202c8 2 m30999| Fri Dec 14 02:59:24.842 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), authoritative: true, shard: "shard0000", shardHost: "localhost:30000" } to localhost:30000 with $auth: {} m30001| Fri Dec 14 02:59:24.843 [initandlisten] connection accepted from 127.0.0.1:38277 #16 (16 connections now open) m30002| Fri Dec 14 02:59:24.843 [initandlisten] connection accepted from 127.0.0.1:40384 #16 (16 connections now open) m30000| Fri Dec 14 02:59:24.842 [conn13] no current chunk manager found for this shard, will initialize m30000| Fri Dec 14 02:59:24.842 [conn13] SyncClusterConnection connecting to [localhost:30000] m30000| Fri Dec 14 02:59:24.842 [initandlisten] connection accepted from 127.0.0.1:48343 #16 (16 connections now open) m30000| Fri Dec 14 02:59:24.843 [conn13] SyncClusterConnection connecting to [localhost:30001] m30000| Fri Dec 14 02:59:24.843 [conn13] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:24.845 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30999| Fri Dec 14 02:59:24.845 [conn1] resetting shard version of test.foo on localhost:30001, version is zero m30999| Fri Dec 14 02:59:24.845 [conn1] have to set shard version for conn: localhost:30001 ns:test.foo my last seq: 0 current: 2 version: 0|0||000000000000000000000000 manager: 0xab26ec0 m30999| Fri Dec 14 02:59:24.845 [conn1] setShardVersion shard0001 localhost:30001 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0001", shardHost: "localhost:30001" } 0xab21928 2 m30999| Fri Dec 14 02:59:24.845 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0001", shardHost: "localhost:30001" } to localhost:30001 with $auth: {} m30999| Fri Dec 14 02:59:24.851 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30999| Fri Dec 14 02:59:24.851 [conn1] resetting shard version of test.foo on localhost:30002, version is zero m30999| Fri Dec 14 02:59:24.851 [conn1] have to set shard version for conn: localhost:30002 ns:test.foo my last seq: 0 current: 2 version: 0|0||000000000000000000000000 manager: 0xab26ec0 m30999| Fri Dec 14 02:59:24.851 [conn1] setShardVersion shard0002 localhost:30002 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0002", shardHost: "localhost:30002" } 0xab22498 2 m30999| Fri Dec 14 02:59:24.851 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b8387ff1af5ba3bb5'), shard: "shard0002", shardHost: "localhost:30002" } to localhost:30002 with $auth: {} m30999| Fri Dec 14 02:59:24.854 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30999| Fri Dec 14 02:59:24.855 [conn1] Request::process end ns: admin.$cmd msg id: 7 op: 2004 attempt: 0 933ms m30999| Fri Dec 14 02:59:24.855 [conn1] Request::process begin ns: config.settings msg id: 8 op: 2001 attempt: 0 m30999| Fri Dec 14 02:59:24.855 [conn1] write: config.settings m30000| Fri Dec 14 02:59:24.855 [conn14] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.856 [conn14] fsync from getlasterror m30001| Fri Dec 14 02:59:24.855 [conn14] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.856 [conn14] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.858 [conn14] fsync from getlasterror m30002| Fri Dec 14 02:59:24.861 [conn14] fsync from getlasterror m30999| Fri Dec 14 02:59:24.862 [conn1] Request::process end ns: config.settings msg id: 8 op: 2001 attempt: 0 6ms m30999| Fri Dec 14 02:59:24.862 [conn1] Request::process begin ns: test.foo msg id: 9 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.862 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.862 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.864 [conn1] about to initiate autosplit: ns:test.fooshard: shard0000:localhost:30000lastmod: 1|0||000000000000000000000000min: { num: MinKey }max: { num: MaxKey } dataWritten: 4655133 splitThreshold: 921 m30999| Fri Dec 14 02:59:24.864 [conn1] chunk not full enough to trigger auto-split no split entry m30999| Fri Dec 14 02:59:24.864 [conn1] Request::process end ns: test.foo msg id: 9 op: 2002 attempt: 0 2ms m30999| Fri Dec 14 02:59:24.864 [conn1] Request::process begin ns: test.foo msg id: 10 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.864 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.864 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.864 [conn1] Request::process end ns: test.foo msg id: 10 op: 2002 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.864 [conn1] Request::process begin ns: test.foo msg id: 11 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.865 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.865 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process end ns: test.foo msg id: 11 op: 2002 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process begin ns: test.foo msg id: 12 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.865 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.865 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process end ns: test.foo msg id: 12 op: 2002 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process begin ns: test.foo msg id: 13 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.865 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.865 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process end ns: test.foo msg id: 13 op: 2002 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process begin ns: test.foo msg id: 14 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.865 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.865 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process end ns: test.foo msg id: 14 op: 2002 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.865 [conn1] Request::process begin ns: test.foo msg id: 15 op: 2002 attempt: 0 m30999| Fri Dec 14 02:59:24.865 [conn1] write: test.foo m30999| Fri Dec 14 02:59:24.865 [conn1] inserting 1 documents to shard shard0000:localhost:30000 at version 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.873 [conn1] about to initiate autosplit: ns:test.fooshard: shard0000:localhost:30000lastmod: 1|0||000000000000000000000000min: { num: MinKey }max: { num: MaxKey } dataWritten: 210 splitThreshold: 921 m30999| Fri Dec 14 02:59:24.873 [conn1] chunk not full enough to trigger auto-split no split entry m30999| Fri Dec 14 02:59:24.873 [conn1] Request::process end ns: test.foo msg id: 15 op: 2002 attempt: 0 7ms m30999| Fri Dec 14 02:59:24.873 [conn1] Request::process begin ns: test.foo msg id: 16 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:24.873 [conn1] shard query: test.foo {} m30999| Fri Dec 14 02:59:24.873 [conn1] creating pcursor over QSpec { ns: "test.foo", n2skip: 0, n2return: 0, options: 0, query: {}, fields: {} } and CInfo { v_ns: "", filter: {} } m30999| Fri Dec 14 02:59:24.874 [conn1] initializing over 1 shards required by [test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7] m30999| Fri Dec 14 02:59:24.874 [conn1] initializing on shard shard0000:localhost:30000, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30999| Fri Dec 14 02:59:24.874 [conn1] initialized query (lazily) on shard shard0000:localhost:30000, current connection state is { state: { conn: "localhost:30000", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: "(empty)", count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false } m30999| Fri Dec 14 02:59:24.874 [conn1] finishing over 1 shards m30999| Fri Dec 14 02:59:24.874 [conn1] finishing on shard shard0000:localhost:30000, current connection state is { state: { conn: "localhost:30000", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: "(empty)", count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false } m30999| Fri Dec 14 02:59:24.875 [conn1] finished on shard shard0000:localhost:30000, current connection state is { state: { conn: "(done)", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: { _id: ObjectId('50cadc5cea322359e0153ef6'), num: 1.0 }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef6'), num: 1.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef7'), num: 2.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef8'), num: 3.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef9'), num: 4.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efa'), num: 5.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efb'), num: 6.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efc'), num: 7.0 } m30999| Fri Dec 14 02:59:24.875 [conn1] hasMore: 0 sendMore: 1 cursorMore: 0 ntoreturn: 0 num: 7 wouldSendMoreIfHad: 1 id:4343435602719 totalSent: 0 m30999| Fri Dec 14 02:59:24.875 [conn1] Request::process end ns: test.foo msg id: 16 op: 2004 attempt: 0 2ms m30998| Fri Dec 14 02:59:24.876 [conn1] DBConfig unserialize: test { _id: "test", partitioned: true, primary: "shard0000" } m30998| Fri Dec 14 02:59:24.876 [conn1] major version query from 0|0||50cadc5c8387ff1af5ba3bb7 and over 0 shards is { ns: "test.foo", $or: [ { lastmod: { $gte: Timestamp 0|0 } } ] } m30998| Fri Dec 14 02:59:24.877 [conn1] found 1 new chunks for collection test.foo (tracking 1), new version is 0xa0a7620 m30998| Fri Dec 14 02:59:24.877 [conn1] loaded 1 chunks into new chunk manager for test.foo with version 1|0||50cadc5c8387ff1af5ba3bb7 m30998| Fri Dec 14 02:59:24.877 [conn1] ChunkManager: time to load chunks for test.foo: 0ms sequenceNumber: 2 version: 1|0||50cadc5c8387ff1af5ba3bb7 based on: (empty) m30998| Fri Dec 14 02:59:24.877 [conn1] found 0 dropped collections and 1 sharded collections for database test m30998| Fri Dec 14 02:59:24.877 [conn1] Request::process begin ns: test.foo msg id: 17 op: 2004 attempt: 0 m30998| Fri Dec 14 02:59:24.877 [conn1] shard query: test.foo {} m30998| Fri Dec 14 02:59:24.877 [conn1] creating pcursor over QSpec { ns: "test.foo", n2skip: 0, n2return: 0, options: 0, query: {}, fields: {} } and CInfo { v_ns: "", filter: {} } m30998| Fri Dec 14 02:59:24.877 [conn1] initializing over 1 shards required by [test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7] m30998| Fri Dec 14 02:59:24.877 [conn1] initializing on shard shard0000:localhost:30000, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30998| Fri Dec 14 02:59:24.877 [conn1] creating new connection to:localhost:30000 m30998| Fri Dec 14 02:59:24.878 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:24.878 [conn1] connected connection! m30998| Fri Dec 14 02:59:24.878 [conn1] creating WriteBackListener for: localhost:30000 serverID: 50cadc5b4db0607719c15419 m30998| Fri Dec 14 02:59:24.878 [conn1] initializing shard connection to localhost:30000 m30998| Fri Dec 14 02:59:24.878 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } m30998| Fri Dec 14 02:59:24.878 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } to localhost:30000 with $auth: {} m30000| Fri Dec 14 02:59:24.878 [initandlisten] connection accepted from 127.0.0.1:48347 #17 (17 connections now open) m30998| Fri Dec 14 02:59:24.878 BackgroundJob starting: WriteBackListener-localhost:30000 m30998| Fri Dec 14 02:59:24.878 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30998| Fri Dec 14 02:59:24.878 [conn1] have to set shard version for conn: localhost:30000 ns:test.foo my last seq: 0 current: 2 version: 1|0||50cadc5c8387ff1af5ba3bb7 manager: 0xa0a7580 m30998| Fri Dec 14 02:59:24.878 [conn1] setShardVersion shard0000 localhost:30000 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0000", shardHost: "localhost:30000" } 0xa0a82e0 2 m30998| Fri Dec 14 02:59:24.878 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 1000|0, versionEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0000", shardHost: "localhost:30000" } to localhost:30000 with $auth: {} m30998| Fri Dec 14 02:59:24.878 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30998| Fri Dec 14 02:59:24.878 [conn1] creating new connection to:localhost:30001 m30998| Fri Dec 14 02:59:24.878 BackgroundJob starting: ConnectBG m30001| Fri Dec 14 02:59:24.879 [initandlisten] connection accepted from 127.0.0.1:38281 #17 (17 connections now open) m30998| Fri Dec 14 02:59:24.879 [conn1] connected connection! m30998| Fri Dec 14 02:59:24.879 [conn1] creating WriteBackListener for: localhost:30001 serverID: 50cadc5b4db0607719c15419 m30998| Fri Dec 14 02:59:24.879 BackgroundJob starting: WriteBackListener-localhost:30001 m30998| Fri Dec 14 02:59:24.879 [conn1] initializing shard connection to localhost:30001 m30998| Fri Dec 14 02:59:24.879 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } m30998| Fri Dec 14 02:59:24.879 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } to localhost:30001 with $auth: {} m30998| Fri Dec 14 02:59:24.879 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30998| Fri Dec 14 02:59:24.879 [conn1] resetting shard version of test.foo on localhost:30001, version is zero m30998| Fri Dec 14 02:59:24.879 [conn1] have to set shard version for conn: localhost:30001 ns:test.foo my last seq: 0 current: 2 version: 0|0||000000000000000000000000 manager: 0xa0a7580 m30998| Fri Dec 14 02:59:24.879 [conn1] setShardVersion shard0001 localhost:30001 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0001", shardHost: "localhost:30001" } 0xa0a9048 2 m30998| Fri Dec 14 02:59:24.879 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0001", shardHost: "localhost:30001" } to localhost:30001 with $auth: {} m30998| Fri Dec 14 02:59:24.879 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30998| Fri Dec 14 02:59:24.879 [conn1] creating new connection to:localhost:30002 m30998| Fri Dec 14 02:59:24.880 BackgroundJob starting: ConnectBG m30002| Fri Dec 14 02:59:24.880 [initandlisten] connection accepted from 127.0.0.1:40388 #17 (17 connections now open) m30998| Fri Dec 14 02:59:24.880 [conn1] connected connection! m30998| Fri Dec 14 02:59:24.880 [conn1] creating WriteBackListener for: localhost:30002 serverID: 50cadc5b4db0607719c15419 m30998| Fri Dec 14 02:59:24.880 BackgroundJob starting: WriteBackListener-localhost:30002 m30998| Fri Dec 14 02:59:24.880 [conn1] initializing shard connection to localhost:30002 m30998| Fri Dec 14 02:59:24.880 [conn1] initial sharding settings : { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } m30998| Fri Dec 14 02:59:24.880 [conn1] Sending command { setShardVersion: "", init: true, configdb: "localhost:30000,localhost:30001,localhost:30002", serverID: ObjectId('50cadc5b4db0607719c15419'), authoritative: true } to localhost:30002 with $auth: {} m30998| Fri Dec 14 02:59:24.880 [conn1] initial sharding result : { initialized: true, ok: 1.0 } m30998| Fri Dec 14 02:59:24.880 [conn1] resetting shard version of test.foo on localhost:30002, version is zero m30998| Fri Dec 14 02:59:24.880 [conn1] have to set shard version for conn: localhost:30002 ns:test.foo my last seq: 0 current: 2 version: 0|0||000000000000000000000000 manager: 0xa0a7580 m30998| Fri Dec 14 02:59:24.880 [conn1] setShardVersion shard0002 localhost:30002 test.foo { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0002", shardHost: "localhost:30002" } 0xa0aa138 2 m30998| Fri Dec 14 02:59:24.881 [conn1] Sending command { setShardVersion: "test.foo", configdb: "localhost:30000,localhost:30001,localhost:30002", version: Timestamp 0|0, versionEpoch: ObjectId('000000000000000000000000'), serverID: ObjectId('50cadc5b4db0607719c15419'), shard: "shard0002", shardHost: "localhost:30002" } to localhost:30002 with $auth: {} m30998| Fri Dec 14 02:59:24.881 [conn1] setShardVersion success: { oldVersion: Timestamp 0|0, oldVersionEpoch: ObjectId('000000000000000000000000'), ok: 1.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] initialized query (lazily) on shard shard0000:localhost:30000, current connection state is { state: { conn: "localhost:30000", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: "(empty)", count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false } m30998| Fri Dec 14 02:59:24.881 [conn1] finishing over 1 shards m30998| Fri Dec 14 02:59:24.881 [conn1] finishing on shard shard0000:localhost:30000, current connection state is { state: { conn: "localhost:30000", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: "(empty)", count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false } m30998| Fri Dec 14 02:59:24.881 [conn1] finished on shard shard0000:localhost:30000, current connection state is { state: { conn: "(done)", vinfo: "test.foo @ 1|0||50cadc5c8387ff1af5ba3bb7", cursor: { _id: ObjectId('50cadc5cea322359e0153ef6'), num: 1.0 }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef6'), num: 1.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef7'), num: 2.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef8'), num: 3.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153ef9'), num: 4.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efa'), num: 5.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efb'), num: 6.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] Matcher::matches() { _id: ObjectId('50cadc5cea322359e0153efc'), num: 7.0 } m30998| Fri Dec 14 02:59:24.881 [conn1] hasMore: 0 sendMore: 1 cursorMore: 0 ntoreturn: 0 num: 7 wouldSendMoreIfHad: 1 id:4063341598435 totalSent: 0 m30999| Fri Dec 14 02:59:24.882 [conn1] Request::process begin ns: admin.$cmd msg id: 18 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:24.882 [conn1] single query: admin.$cmd { split: "test.foo", middle: { num: 4.0 } } ntoreturn: -1 options : 0 m30998| Fri Dec 14 02:59:24.882 [conn1] Request::process end ns: test.foo msg id: 17 op: 2004 attempt: 0 4ms m30999| Fri Dec 14 02:59:24.882 [conn1] splitting: test.foo shard: ns:test.fooshard: shard0000:localhost:30000lastmod: 1|0||000000000000000000000000min: { num: MinKey }max: { num: MaxKey } m30000| Fri Dec 14 02:59:24.883 [conn15] received splitChunk request: { splitChunk: "test.foo", keyPattern: { num: 1.0 }, min: { num: MinKey }, max: { num: MaxKey }, from: "shard0000", splitKeys: [ { num: 4.0 } ], shardId: "test.foo-num_MinKey", configdb: "localhost:30000,localhost:30001,localhost:30002" } m30000| Fri Dec 14 02:59:24.883 [initandlisten] connection accepted from 127.0.0.1:48350 #18 (18 connections now open) m30001| Fri Dec 14 02:59:24.884 [initandlisten] connection accepted from 127.0.0.1:38284 #18 (18 connections now open) m30002| Fri Dec 14 02:59:24.884 [initandlisten] connection accepted from 127.0.0.1:40391 #18 (18 connections now open) m30000| Fri Dec 14 02:59:24.887 [conn15] SyncClusterConnection connecting to [localhost:30000] m30000| Fri Dec 14 02:59:24.887 [LockPinger] creating distributed lock ping thread for localhost:30000,localhost:30001,localhost:30002 and process bs-rhel-57-32-1:30000:1355471964:356303735 (sleeping for 30000ms) m30000| Fri Dec 14 02:59:24.887 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.887 [initandlisten] connection accepted from 127.0.0.1:48353 #19 (19 connections now open) m30000| Fri Dec 14 02:59:24.887 [conn15] SyncClusterConnection connecting to [localhost:30001] m30001| Fri Dec 14 02:59:24.887 [initandlisten] connection accepted from 127.0.0.1:38287 #19 (19 connections now open) m30000| Fri Dec 14 02:59:24.887 [conn15] SyncClusterConnection connecting to [localhost:30002] m30002| Fri Dec 14 02:59:24.888 [initandlisten] connection accepted from 127.0.0.1:40394 #19 (19 connections now open) m30000| Fri Dec 14 02:59:24.888 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.891 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.891 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.892 [conn19] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.892 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.892 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.894 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:24.898 [conn19] fsync from getlasterror m30001| Fri Dec 14 02:59:24.898 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.901 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.902 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.908 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.908 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.908 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.909 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.909 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.909 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.909 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.909 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:24.910 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:24.910 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:24.912 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.913 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.913 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:24.917 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.917 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.920 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.920 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:24.920 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.920 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.920 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.920 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.921 [conn19] fsync from getlasterror m30001| Fri Dec 14 02:59:24.923 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:24.925 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.926 [conn15] distributed lock 'test.foo/bs-rhel-57-32-1:30000:1355471964:356303735' acquired, ts : 50cadc5c563eed1281c1adf9 m30000| Fri Dec 14 02:59:24.927 [conn15] splitChunk accepted at version 1|0||50cadc5c8387ff1af5ba3bb7 m30000| Fri Dec 14 02:59:24.927 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.928 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.928 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.929 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:24.935 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.940 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:24.945 [conn15] about to log metadata event: { _id: "bs-rhel-57-32-1-2012-12-14T07:59:24-0", server: "bs-rhel-57-32-1", clientAddr: "127.0.0.1:48339", time: new Date(1355471964945), what: "split", ns: "test.foo", details: { before: { min: { num: MinKey }, max: { num: MaxKey }, lastmod: Timestamp 1000|0, lastmodEpoch: ObjectId('000000000000000000000000') }, left: { min: { num: MinKey }, max: { num: 4.0 }, lastmod: Timestamp 1000|1, lastmodEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7') }, right: { min: { num: 4.0 }, max: { num: MaxKey }, lastmod: Timestamp 1000|2, lastmodEpoch: ObjectId('50cadc5c8387ff1af5ba3bb7') } } } m30000| Fri Dec 14 02:59:24.945 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.945 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.945 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.945 [conn16] build index config.changelog { _id: 1 } m30000| Fri Dec 14 02:59:24.946 [conn16] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:24.946 [conn16] build index config.changelog { _id: 1 } m30001| Fri Dec 14 02:59:24.947 [conn16] build index done. scanned 0 total records. 0 secs m30002| Fri Dec 14 02:59:24.947 [conn16] build index config.changelog { _id: 1 } m30002| Fri Dec 14 02:59:24.947 [conn16] build index done. scanned 0 total records. 0 secs m30000| Fri Dec 14 02:59:24.947 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:24.954 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.958 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:24.963 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.963 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.963 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.964 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:24.968 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:24.971 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:24.977 [conn19] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.978 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.978 [conn19] fsync from getlasterror m30001| Fri Dec 14 02:59:24.977 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.980 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:24.982 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.983 [conn15] distributed lock 'test.foo/bs-rhel-57-32-1:30000:1355471964:356303735' unlocked. m30999| Fri Dec 14 02:59:24.984 [conn1] loading chunk manager for collection test.foo using old chunk manager w/ version 1|0||50cadc5c8387ff1af5ba3bb7 and 1 chunks m30999| Fri Dec 14 02:59:24.984 [conn1] major version query from 1|0||50cadc5c8387ff1af5ba3bb7 and over 1 shards is { ns: "test.foo", $or: [ { lastmod: { $gte: Timestamp 1000|0 } }, { shard: "shard0000", lastmod: { $gt: Timestamp 1000|0 } } ] } m30999| Fri Dec 14 02:59:24.984 [conn1] found 2 new chunks for collection test.foo (tracking 2), new version is 0xab27328 m30999| Fri Dec 14 02:59:24.984 [conn1] loaded 2 chunks into new chunk manager for test.foo with version 1|2||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.984 [conn1] ChunkManager: time to load chunks for test.foo: 0ms sequenceNumber: 3 version: 1|2||50cadc5c8387ff1af5ba3bb7 based on: 1|0||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:24.984 [conn1] Request::process end ns: admin.$cmd msg id: 18 op: 2004 attempt: 0 102ms m30999| Fri Dec 14 02:59:24.989 [conn1] Request::process begin ns: config.databases msg id: 19 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:24.989 [conn1] shard query: config.databases { _id: "test" } m30999| Fri Dec 14 02:59:24.989 [conn1] creating pcursor over QSpec { ns: "config.databases", n2skip: 0, n2return: -1, options: 0, query: { _id: "test" }, fields: {} } and CInfo { v_ns: "", filter: {} } m30999| Fri Dec 14 02:59:24.989 [conn1] initializing over 1 shards required by [unsharded @ config:localhost:30000,localhost:30001,localhost:30002] m30999| Fri Dec 14 02:59:24.989 [conn1] initializing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] initialized query (full) on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "test", partitioned: true, primary: "shard0000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] finishing over 1 shards m30999| Fri Dec 14 02:59:24.989 [conn1] finishing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "test", partitioned: true, primary: "shard0000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] finished on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "(done)", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "test", partitioned: true, primary: "shard0000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] Request::process end ns: config.databases msg id: 19 op: 2004 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.989 [conn1] Request::process begin ns: config.shards msg id: 20 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:24.989 [conn1] shard query: config.shards { _id: "shard0000" } m30999| Fri Dec 14 02:59:24.989 [conn1] creating pcursor over QSpec { ns: "config.shards", n2skip: 0, n2return: -1, options: 0, query: { _id: "shard0000" }, fields: {} } and CInfo { v_ns: "", filter: {} } m30999| Fri Dec 14 02:59:24.989 [conn1] initializing over 1 shards required by [unsharded @ config:localhost:30000,localhost:30001,localhost:30002] m30999| Fri Dec 14 02:59:24.989 [conn1] initializing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] initialized query (full) on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "shard0000", host: "localhost:30000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] finishing over 1 shards m30999| Fri Dec 14 02:59:24.989 [conn1] finishing on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "SyncClusterConnection [localhost:30000,localhost:30001,localhost:30002]", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "shard0000", host: "localhost:30000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] finished on shard config:localhost:30000,localhost:30001,localhost:30002, current connection state is { state: { conn: "(done)", vinfo: "config:localhost:30000,localhost:30001,localhost:30002", cursor: { _id: "shard0000", host: "localhost:30000" }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false } m30999| Fri Dec 14 02:59:24.989 [conn1] Request::process end ns: config.shards msg id: 20 op: 2004 attempt: 0 0ms m30999| Fri Dec 14 02:59:24.990 [conn1] Request::process begin ns: admin.$cmd msg id: 21 op: 2004 attempt: 0 m30999| Fri Dec 14 02:59:24.990 [conn1] single query: admin.$cmd { movechunk: "test.foo", find: { num: 3.0 }, to: "localhost:30001" } ntoreturn: -1 options : 0 m30999| Fri Dec 14 02:59:24.990 [conn1] CMD: movechunk: { movechunk: "test.foo", find: { num: 3.0 }, to: "localhost:30001" } m30999| Fri Dec 14 02:59:24.990 [conn1] moving chunk ns: test.foo moving ( ns:test.fooshard: shard0000:localhost:30000lastmod: 1|1||000000000000000000000000min: { num: MinKey }max: { num: 4.0 }) shard0000:localhost:30000 -> shard0001:localhost:30001 m30000| Fri Dec 14 02:59:24.990 [conn15] received moveChunk request: { moveChunk: "test.foo", from: "localhost:30000", to: "localhost:30001", fromShard: "shard0000", toShard: "shard0001", min: { num: MinKey }, max: { num: 4.0 }, maxChunkSizeBytes: 52428800, shardId: "test.foo-num_MinKey", configdb: "localhost:30000,localhost:30001,localhost:30002", secondaryThrottle: false, waitForDelete: false } m30000| Fri Dec 14 02:59:24.991 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.991 [conn19] fsync from getlasterror m30001| Fri Dec 14 02:59:24.991 [conn19] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.991 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.994 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:24.996 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:24.997 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:24.997 [conn19] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:24.997 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:24.997 [conn19] fsync from getlasterror m30001| Fri Dec 14 02:59:25.000 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:25.002 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:25.003 [conn15] distributed lock 'test.foo/bs-rhel-57-32-1:30000:1355471964:356303735' acquired, ts : 50cadc5c563eed1281c1adfa m30000| Fri Dec 14 02:59:25.003 [conn15] about to log metadata event: { _id: "bs-rhel-57-32-1-2012-12-14T07:59:25-1", server: "bs-rhel-57-32-1", clientAddr: "127.0.0.1:48339", time: new Date(1355471965003), what: "moveChunk.start", ns: "test.foo", details: { min: { num: MinKey }, max: { num: 4.0 }, from: "shard0000", to: "shard0001" } } m30000| Fri Dec 14 02:59:25.003 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:25.003 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:25.003 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:25.004 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:25.007 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:25.010 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:25.013 [conn15] moveChunk request accepted at version 1|2||50cadc5c8387ff1af5ba3bb7 m30000| Fri Dec 14 02:59:25.013 [conn15] moveChunk number of documents: 3 m30000| Fri Dec 14 02:59:25.014 [initandlisten] connection accepted from 127.0.0.1:48357 #20 (20 connections now open) m30001| Fri Dec 14 02:59:25.014 [FileAllocator] allocating new datafile /data/db/sync21/test.ns, filling with zeroes... m30000| Fri Dec 14 02:59:25.044 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.048 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30001| Fri Dec 14 02:59:25.055 [FileAllocator] done allocating datafile /data/db/sync21/test.ns, size: 16MB, took 0.04 secs m30001| Fri Dec 14 02:59:25.055 [FileAllocator] allocating new datafile /data/db/sync21/test.0, filling with zeroes... m30000| Fri Dec 14 02:59:25.056 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.068 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.088 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.124 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.192 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "ready", counts: { cloned: 0, clonedBytes: 0, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30001| Fri Dec 14 02:59:25.281 [FileAllocator] done allocating datafile /data/db/sync21/test.0, size: 16MB, took 0.226 secs m30001| Fri Dec 14 02:59:25.282 [FileAllocator] allocating new datafile /data/db/sync21/test.1, filling with zeroes... m30001| Fri Dec 14 02:59:25.284 [migrateThread] build index test.foo { _id: 1 } m30001| Fri Dec 14 02:59:25.285 [migrateThread] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:25.285 [migrateThread] info: creating collection test.foo on add index m30001| Fri Dec 14 02:59:25.285 [migrateThread] build index test.foo { num: 1.0 } m30001| Fri Dec 14 02:59:25.286 [migrateThread] build index done. scanned 0 total records. 0 secs m30001| Fri Dec 14 02:59:25.287 [migrateThread] Waiting for replication to catch up before entering critical section m30001| Fri Dec 14 02:59:25.287 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.foo' { num: MinKey } -> { num: 4.0 } m30000| Fri Dec 14 02:59:25.324 [conn15] moveChunk data transfer progress: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "steady", counts: { cloned: 3, clonedBytes: 105, catchup: 0, steady: 0 }, ok: 1.0 } my mem used: 0 m30000| Fri Dec 14 02:59:25.324 [conn15] moveChunk setting version to: 2|0||50cadc5c8387ff1af5ba3bb7 m30001| Fri Dec 14 02:59:25.325 [initandlisten] connection accepted from 127.0.0.1:38293 #20 (20 connections now open) m30001| Fri Dec 14 02:59:25.325 [conn20] Waiting for commit to finish m30001| Fri Dec 14 02:59:25.328 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.foo' { num: MinKey } -> { num: 4.0 } m30001| Fri Dec 14 02:59:25.328 [migrateThread] about to log metadata event: { _id: "bs-rhel-57-32-1-2012-12-14T07:59:25-0", server: "bs-rhel-57-32-1", clientAddr: ":27017", time: new Date(1355471965328), what: "moveChunk.to", ns: "test.foo", details: { min: { num: MinKey }, max: { num: 4.0 }, step1 of 5: 272, step2 of 5: 0, step3 of 5: 0, step4 of 5: 0, step5 of 5: 40 } } m30001| Fri Dec 14 02:59:25.328 [migrateThread] SyncClusterConnection connecting to [localhost:30000] m30000| Fri Dec 14 02:59:25.332 [initandlisten] connection accepted from 127.0.0.1:48361 #21 (21 connections now open) m30001| Fri Dec 14 02:59:25.332 [migrateThread] SyncClusterConnection connecting to [localhost:30001] m30001| Fri Dec 14 02:59:25.332 [initandlisten] connection accepted from 127.0.0.1:38295 #21 (21 connections now open) m30001| Fri Dec 14 02:59:25.332 [migrateThread] SyncClusterConnection connecting to [localhost:30002] m30000| Fri Dec 14 02:59:25.333 [conn15] moveChunk migrate commit accepted by TO-shard: { active: true, ns: "test.foo", from: "localhost:30000", min: { num: MinKey }, max: { num: 4.0 }, shardKeyPattern: { num: 1.0 }, state: "done", counts: { cloned: 3, clonedBytes: 105, catchup: 0, steady: 0 }, ok: 1.0 } m30000| Fri Dec 14 02:59:25.333 [conn15] moveChunk updating self version to: 2|1||50cadc5c8387ff1af5ba3bb7 through { num: 4.0 } -> { num: MaxKey } for collection 'test.foo' m30000| Fri Dec 14 02:59:25.336 [conn15] SyncClusterConnection connecting to [localhost:30000] m30002| Fri Dec 14 02:59:25.337 [initandlisten] connection accepted from 127.0.0.1:40402 #20 (20 connections now open) m30000| Fri Dec 14 02:59:25.337 [conn21] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:25.337 [conn21] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:25.338 [initandlisten] connection accepted from 127.0.0.1:48364 #22 (22 connections now open) m30000| Fri Dec 14 02:59:25.338 [conn15] SyncClusterConnection connecting to [localhost:30001] m30001| Fri Dec 14 02:59:25.338 [initandlisten] connection accepted from 127.0.0.1:38298 #22 (22 connections now open) m30000| Fri Dec 14 02:59:25.339 [conn15] SyncClusterConnection connecting to [localhost:30002] m30002| Fri Dec 14 02:59:25.339 [initandlisten] connection accepted from 127.0.0.1:40405 #21 (21 connections now open) m30000| Fri Dec 14 02:59:25.342 [conn22] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:25.343 [conn22] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:25.915 [FileAllocator] done allocating datafile /data/db/sync21/test.1, size: 32MB, took 0.632 secs m30001| Fri Dec 14 02:59:26.549 [conn22] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:1 reslen:51 1206ms m30001| Fri Dec 14 02:59:26.550 [conn21] command admin.$cmd command: { fsync: 1 } ntoreturn:1 keyUpdates:0 locks(micros) W:1 reslen:51 1212ms m30002| Fri Dec 14 02:59:26.550 [conn21] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:26.550 [conn20] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:26.552 [conn22] fsync from getlasterror m30000| Fri Dec 14 02:59:26.552 [conn21] fsync from getlasterror m30001| Fri Dec 14 02:59:26.556 [conn22] fsync from getlasterror m30001| Fri Dec 14 02:59:26.556 [conn21] fsync from getlasterror m30002| Fri Dec 14 02:59:26.559 [conn20] fsync from getlasterror m30002| Fri Dec 14 02:59:26.559 [conn21] fsync from getlasterror m30000| Fri Dec 14 02:59:26.562 [conn15] about to log metadata event: { _id: "bs-rhel-57-32-1-2012-12-14T07:59:26-2", server: "bs-rhel-57-32-1", clientAddr: "127.0.0.1:48339", time: new Date(1355471966562), what: "moveChunk.commit", ns: "test.foo", details: { min: { num: MinKey }, max: { num: 4.0 }, from: "shard0000", to: "shard0001" } } m30000| Fri Dec 14 02:59:26.563 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:26.563 [conn21] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:26.563 [conn21] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:26.563 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:26.563 [conn20] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:26.563 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:26.564 [conn21] fsync from getlasterror m30000| Fri Dec 14 02:59:26.564 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:26.568 [conn21] fsync from getlasterror m30001| Fri Dec 14 02:59:26.568 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:26.571 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:26.571 [conn20] fsync from getlasterror m30000| Fri Dec 14 02:59:26.574 [conn15] MigrateFromStatus::done About to acquire global write lock to exit critical section m30000| Fri Dec 14 02:59:26.575 [conn15] MigrateFromStatus::done Global lock acquired m30000| Fri Dec 14 02:59:26.575 [conn15] forking for cleanup of chunk data m30000| Fri Dec 14 02:59:26.575 [conn15] MigrateFromStatus::done About to acquire global write lock to exit critical section m30000| Fri Dec 14 02:59:26.575 [conn15] MigrateFromStatus::done Global lock acquired m30000| Fri Dec 14 02:59:26.575 [conn19] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:26.575 [conn19] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:26.575 [conn19] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:26.576 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:26.576 [cleanupOldData-50cadc5e563eed1281c1adfb] (start) waiting to cleanup test.foo from { num: MinKey } -> { num: 4.0 }, # cursors remaining: 0 m30001| Fri Dec 14 02:59:26.579 [conn19] fsync from getlasterror m30002| Fri Dec 14 02:59:26.581 [conn19] fsync from getlasterror m30000| Fri Dec 14 02:59:26.582 [conn15] distributed lock 'test.foo/bs-rhel-57-32-1:30000:1355471964:356303735' unlocked. m30000| Fri Dec 14 02:59:26.582 [conn15] about to log metadata event: { _id: "bs-rhel-57-32-1-2012-12-14T07:59:26-3", server: "bs-rhel-57-32-1", clientAddr: "127.0.0.1:48339", time: new Date(1355471966582), what: "moveChunk.from", ns: "test.foo", details: { min: { num: MinKey }, max: { num: 4.0 }, step1 of 6: 0, step2 of 6: 22, step3 of 6: 0, step4 of 6: 310, step5 of 6: 1250, step6 of 6: 0 } } m30000| Fri Dec 14 02:59:26.583 [conn16] CMD fsync: sync:1 lock:0 m30001| Fri Dec 14 02:59:26.583 [conn16] CMD fsync: sync:1 lock:0 m30002| Fri Dec 14 02:59:26.583 [conn16] CMD fsync: sync:1 lock:0 m30000| Fri Dec 14 02:59:26.583 [conn16] fsync from getlasterror m30001| Fri Dec 14 02:59:26.587 [conn16] fsync from getlasterror m30002| Fri Dec 14 02:59:26.590 [conn16] fsync from getlasterror m30000| Fri Dec 14 02:59:26.592 [conn15] command admin.$cmd command: { moveChunk: "test.foo", from: "localhost:30000", to: "localhost:30001", fromShard: "shard0000", toShard: "shard0001", min: { num: MinKey }, max: { num: 4.0 }, maxChunkSizeBytes: 52428800, shardId: "test.foo-num_MinKey", configdb: "localhost:30000,localhost:30001,localhost:30002", secondaryThrottle: false, waitForDelete: false } ntoreturn:1 keyUpdates:0 locks(micros) W:23 r:56 w:22 reslen:37 1601ms m30999| Fri Dec 14 02:59:26.592 [conn1] moveChunk result: { ok: 1.0 } m30999| Fri Dec 14 02:59:26.593 [conn1] loading chunk manager for collection test.foo using old chunk manager w/ version 1|2||50cadc5c8387ff1af5ba3bb7 and 2 chunks m30999| Fri Dec 14 02:59:26.593 [conn1] major version query from 1|2||50cadc5c8387ff1af5ba3bb7 and over 1 shards is { ns: "test.foo", $or: [ { lastmod: { $gte: Timestamp 1000|2 } }, { shard: "shard0000", lastmod: { $gt: Timestamp 1000|2 } } ] } m30999| Fri Dec 14 02:59:26.595 [conn1] found 2 new chunks for collection test.foo (tracking 2), new version is 0xab25140 m30999| Fri Dec 14 02:59:26.595 [conn1] loaded 2 chunks into new chunk manager for test.foo with version 2|1||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:26.595 [conn1] ChunkManager: time to load chunks for test.foo: 2ms sequenceNumber: 4 version: 2|1||50cadc5c8387ff1af5ba3bb7 based on: 1|2||50cadc5c8387ff1af5ba3bb7 m30999| Fri Dec 14 02:59:26.596 [conn1] Request::process end ns: admin.$cmd msg id: 21 op: 2004 attempt: 0 1606ms assert: [7] != [10] are not equal : combined shards Error: Printing Stack Trace at printStackTrace (src/mongo/shell/utils.js:37:7) at doassert (src/mongo/shell/utils.js:58:1) at Function.assert.eq (src/mongo/shell/utils.js:88:1) at /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/jstests/sharding/sync2.js:30:8 m30000| Fri Dec 14 02:59:26.603 [cleanupOldData-50cadc5e563eed1281c1adfb] waiting to remove documents for test.foo from { num: MinKey } -> { num: 4.0 } Fri Dec 14 02:59:26.605 exec error: src/mongo/shell/utils.js:59 [7] != [10] are not equal : combined shards throw msg; ^ failed to load: /data/buildslaves/Linux_RHEL_32bit_Nightly/mongo/jstests/sharding/sync2.js m30000| Fri Dec 14 02:59:26.605 [cleanupOldData-50cadc5e563eed1281c1adfb] moveChunk starting delete for: test.foo from { num: MinKey } -> { num: 4.0 } m30000| Fri Dec 14 02:59:26.605 got signal 15 (Terminated), will terminate after current cmd ends m30000| Fri Dec 14 02:59:26.606 [interruptThread] now exiting m30000| Fri Dec 14 02:59:26.606 dbexit: m30000| Fri Dec 14 02:59:26.606 [interruptThread] shutdown: going to close listening sockets... m30000| Fri Dec 14 02:59:26.606 [interruptThread] closing listening socket: 14 m30000| Fri Dec 14 02:59:26.606 [interruptThread] closing listening socket: 15 m30000| Fri Dec 14 02:59:26.606 [interruptThread] closing listening socket: 16 m30000| Fri Dec 14 02:59:26.606 [interruptThread] removing socket file: /tmp/mongodb-30000.sock m30000| Fri Dec 14 02:59:26.606 [interruptThread] shutdown: going to flush diaglog... m30000| Fri Dec 14 02:59:26.606 [interruptThread] shutdown: going to close sockets... m30000| Fri Dec 14 02:59:26.606 [interruptThread] shutdown: waiting for fs preallocator... m30998| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30000 m30998| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [0] server [127.0.0.1:30000] m30998| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30999| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30000 m30999| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [0] server [127.0.0.1:30000] m30999| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:26.606 [WriteBackListener-localhost:30000] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30001| Fri Dec 14 02:59:26.606 [conn16] end connection 127.0.0.1:38277 (21 connections now open) m30002| Fri Dec 14 02:59:26.607 [conn16] end connection 127.0.0.1:40384 (20 connections now open) m30002| Fri Dec 14 02:59:26.607 [conn18] end connection 127.0.0.1:40391 (19 connections now open) m30001| Fri Dec 14 02:59:26.607 [conn18] end connection 127.0.0.1:38284 (20 connections now open) m30001| Fri Dec 14 02:59:26.607 [conn19] end connection 127.0.0.1:38287 (19 connections now open) m30002| Fri Dec 14 02:59:26.607 [conn19] end connection 127.0.0.1:40394 (18 connections now open) m30001| Fri Dec 14 02:59:26.608 [conn20] end connection 127.0.0.1:38293 (18 connections now open) m30001| Fri Dec 14 02:59:26.608 [conn22] end connection 127.0.0.1:38298 (18 connections now open) m30000| Fri Dec 14 02:59:26.606 [interruptThread] shutdown: closing all files... m30000| Fri Dec 14 02:59:26.607 [conn16] end connection 127.0.0.1:48343 (21 connections now open) m30000| Fri Dec 14 02:59:26.607 [conn18] end connection 127.0.0.1:48350 (21 connections now open) m30000| Fri Dec 14 02:59:26.607 [conn19] end connection 127.0.0.1:48353 (21 connections now open) m30000| Fri Dec 14 02:59:26.607 [conn22] end connection 127.0.0.1:48364 (21 connections now open) m30002| Fri Dec 14 02:59:26.608 [conn21] end connection 127.0.0.1:40405 (17 connections now open) m30000| Fri Dec 14 02:59:26.621 [interruptThread] closeAllFiles() finished m30000| Fri Dec 14 02:59:26.621 [interruptThread] shutdown: removing fs lock... m30000| Fri Dec 14 02:59:26.622 dbexit: really exiting now m30998| Fri Dec 14 02:59:26.622 [WriteBackListener-localhost:30000] Detecting bad connection created at 0 microSec, clearing pool for localhost:30000 m30998| Fri Dec 14 02:59:26.622 [WriteBackListener-localhost:30000] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30999| Fri Dec 14 02:59:26.622 [WriteBackListener-localhost:30000] Detecting bad connection created at 0 microSec, clearing pool for localhost:30000 m30999| Fri Dec 14 02:59:26.622 [WriteBackListener-localhost:30000] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30001| Fri Dec 14 02:59:27.607 got signal 15 (Terminated), will terminate after current cmd ends m30001| Fri Dec 14 02:59:27.607 [interruptThread] now exiting m30001| Fri Dec 14 02:59:27.607 dbexit: m30001| Fri Dec 14 02:59:27.607 [interruptThread] shutdown: going to close listening sockets... m30001| Fri Dec 14 02:59:27.607 [interruptThread] closing listening socket: 17 m30001| Fri Dec 14 02:59:27.607 [interruptThread] closing listening socket: 18 m30001| Fri Dec 14 02:59:27.607 [interruptThread] closing listening socket: 19 m30001| Fri Dec 14 02:59:27.607 [interruptThread] removing socket file: /tmp/mongodb-30001.sock m30001| Fri Dec 14 02:59:27.607 [interruptThread] shutdown: going to flush diaglog... m30001| Fri Dec 14 02:59:27.607 [interruptThread] shutdown: going to close sockets... m30001| Fri Dec 14 02:59:27.607 [interruptThread] shutdown: waiting for fs preallocator... m30001| Fri Dec 14 02:59:27.607 [interruptThread] shutdown: closing all files... m30002| Fri Dec 14 02:59:27.615 [conn20] end connection 127.0.0.1:40402 (16 connections now open) m30001| Fri Dec 14 02:59:27.615 [conn21] end connection 127.0.0.1:38295 (16 connections now open) m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] Socket recv() conn closed? 127.0.0.1:30001 m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] SocketException: remote: 127.0.0.1:30001 error: 9001 socket exception [0] server [127.0.0.1:30001] m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] Detecting bad connection created at 0 microSec, clearing pool for localhost:30001 m30998| Fri Dec 14 02:59:27.615 [WriteBackListener-localhost:30001] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] Socket recv() conn closed? 127.0.0.1:30001 m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] SocketException: remote: 127.0.0.1:30001 error: 9001 socket exception [0] server [127.0.0.1:30001] m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] Detecting bad connection created at 0 microSec, clearing pool for localhost:30001 m30999| Fri Dec 14 02:59:27.618 [WriteBackListener-localhost:30001] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30001| Fri Dec 14 02:59:27.618 [conn15] end connection 127.0.0.1:38273 (15 connections now open) m30001| Fri Dec 14 02:59:27.622 [interruptThread] closeAllFiles() finished m30001| Fri Dec 14 02:59:27.622 [interruptThread] shutdown: removing fs lock... m30001| Fri Dec 14 02:59:27.622 dbexit: really exiting now m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30000 m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [0] server [127.0.0.1:30000] m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] query failed to: localhost:30000 no data m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30001 m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30001 error: 9001 socket exception [0] server [127.0.0.1:30001] m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] query failed to: localhost:30001 no data m30998| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] creating new connection to:localhost:30000 m30998| Fri Dec 14 02:59:27.623 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30000 m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [0] server [127.0.0.1:30000] m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] query failed to: localhost:30000 no data m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] Socket recv() conn closed? 127.0.0.1:30001 m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30001 error: 9001 socket exception [0] server [127.0.0.1:30001] m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:27.623 [WriteBackListener-localhost:30000] query failed to: localhost:30001 no data m30999| Fri Dec 14 02:59:27.635 [WriteBackListener-localhost:30000] creating new connection to:localhost:30000 m30999| Fri Dec 14 02:59:27.635 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:27.638 [WriteBackListener-localhost:30000] WriteBackListener exception : socket exception [CONNECT_ERROR] for localhost:30000 m30999| Fri Dec 14 02:59:27.638 [WriteBackListener-localhost:30000] WriteBackListener exception : socket exception [CONNECT_ERROR] for localhost:30000 m30998| Fri Dec 14 02:59:28.618 [WriteBackListener-localhost:30001] trying reconnect to localhost:30000 m30998| Fri Dec 14 02:59:28.618 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:28.618 [WriteBackListener-localhost:30001] reconnect localhost:30000 failed couldn't connect to server localhost:30000 m30998| Fri Dec 14 02:59:28.618 [WriteBackListener-localhost:30001] query failed to: localhost:30000 exception: socket exception [CONNECT_ERROR] for localhost:30000 m30998| Fri Dec 14 02:59:28.618 [WriteBackListener-localhost:30001] trying reconnect to localhost:30001 m30002| Fri Dec 14 02:59:28.610 got signal 15 (Terminated), will terminate after current cmd ends m30002| Fri Dec 14 02:59:28.610 [interruptThread] now exiting m30002| Fri Dec 14 02:59:28.610 dbexit: m30002| Fri Dec 14 02:59:28.610 [interruptThread] shutdown: going to close listening sockets... m30002| Fri Dec 14 02:59:28.610 [interruptThread] closing listening socket: 20 m30002| Fri Dec 14 02:59:28.610 [interruptThread] closing listening socket: 21 m30002| Fri Dec 14 02:59:28.610 [interruptThread] closing listening socket: 22 m30002| Fri Dec 14 02:59:28.610 [interruptThread] removing socket file: /tmp/mongodb-30002.sock m30002| Fri Dec 14 02:59:28.610 [interruptThread] shutdown: going to flush diaglog... m30002| Fri Dec 14 02:59:28.610 [interruptThread] shutdown: going to close sockets... m30002| Fri Dec 14 02:59:28.610 [interruptThread] shutdown: waiting for fs preallocator... m30002| Fri Dec 14 02:59:28.610 [interruptThread] shutdown: closing all files... m30002| Fri Dec 14 02:59:28.621 [interruptThread] closeAllFiles() finished m30002| Fri Dec 14 02:59:28.621 [interruptThread] shutdown: removing fs lock... m30002| Fri Dec 14 02:59:28.621 dbexit: really exiting now m30998| Fri Dec 14 02:59:28.625 BackgroundJob starting: ConnectBG m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30001] reconnect localhost:30001 failed couldn't connect to server localhost:30001 m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30001] query failed to: localhost:30001 exception: socket exception [CONNECT_ERROR] for localhost:30001 m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] Socket recv() conn closed? 127.0.0.1:30002 m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [0] server [127.0.0.1:30002] m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30002 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] Detecting bad connection created at 0 microSec, clearing pool for localhost:30002 m30999| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30002 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b8387ff1af5ba3bb5') } m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] Socket recv() conn closed? 127.0.0.1:30002 m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [0] server [127.0.0.1:30002] m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] User Assertion: 10276:DBClientBase::findN: transport error: localhost:30002 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30002] Detecting bad connection created at 0 microSec, clearing pool for localhost:30002 m30998| Fri Dec 14 02:59:28.625 [WriteBackListener-localhost:30001] Socket recv() errno:104 Connection reset by peer 127.0.0.1:30002 m30998| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [1] server [127.0.0.1:30002] m30998| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed m30998| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] query failed to: localhost:30002 no data m30998| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30002] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30002 ns: admin.$cmd query: { writebacklisten: ObjectId('50cadc5b4db0607719c15419') } m30999| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] trying reconnect to localhost:30000 m30999| Fri Dec 14 02:59:28.626 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] reconnect localhost:30000 failed couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] query failed to: localhost:30000 exception: socket exception [CONNECT_ERROR] for localhost:30000 m30999| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] trying reconnect to localhost:30001 m30999| Fri Dec 14 02:59:28.626 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:28.626 [WriteBackListener-localhost:30001] reconnect localhost:30001 failed couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:28.627 [WriteBackListener-localhost:30001] query failed to: localhost:30001 exception: socket exception [CONNECT_ERROR] for localhost:30001 m30999| Fri Dec 14 02:59:28.627 [WriteBackListener-localhost:30001] Socket recv() conn closed? 127.0.0.1:30002 m30999| Fri Dec 14 02:59:28.627 [WriteBackListener-localhost:30001] SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [0] server [127.0.0.1:30002] m30999| Fri Dec 14 02:59:28.627 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:28.627 [WriteBackListener-localhost:30001] query failed to: localhost:30002 no data m30999| Fri Dec 14 02:59:28.638 [WriteBackListener-localhost:30001] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30999| Fri Dec 14 02:59:28.638 [WriteBackListener-localhost:30001] WriteBackListener exception : all servers down/unreachable when querying: localhost:30000,localhost:30001,localhost:30002 m30998| Fri Dec 14 02:59:28.638 [WriteBackListener-localhost:30001] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30998| Fri Dec 14 02:59:28.638 [WriteBackListener-localhost:30001] WriteBackListener exception : all servers down/unreachable when querying: localhost:30000,localhost:30001,localhost:30002 m30998| Fri Dec 14 02:59:29.614 [mongosMain] dbexit: received signal 15 rc:0 received signal 15 m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] Socket recv() conn closed? 127.0.0.1:30000 m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [0] server [127.0.0.1:30000] m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] query failed to: localhost:30000 no data m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] Socket recv() conn closed? 127.0.0.1:30001 m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] SocketException: remote: 127.0.0.1:30001 error: 9001 socket exception [0] server [127.0.0.1:30001] m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] query failed to: localhost:30001 no data m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] Socket recv() conn closed? 127.0.0.1:30002 m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [0] server [127.0.0.1:30002] m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] DBClientCursor::init call() failed m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] query failed to: localhost:30002 no data m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30999| Fri Dec 14 02:59:29.630 [WriteBackListener-localhost:30002] WriteBackListener exception : all servers down/unreachable when querying: localhost:30000,localhost:30001,localhost:30002 m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:29.642 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connect fail to: localhost:30000 errmsg: couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:29.642 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connect fail to: localhost:30001 errmsg: couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:29.642 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] SyncClusterConnection connect fail to: localhost:30002 errmsg: couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] trying reconnect to localhost:30000 m30999| Fri Dec 14 02:59:29.642 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.642 [WriteBackListener-localhost:30000] reconnect localhost:30000 failed couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] query failed to: localhost:30000 exception: socket exception [CONNECT_ERROR] for localhost:30000 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] trying reconnect to localhost:30001 m30999| Fri Dec 14 02:59:29.643 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] reconnect localhost:30001 failed couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] query failed to: localhost:30001 exception: socket exception [CONNECT_ERROR] for localhost:30001 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] trying reconnect to localhost:30002 m30999| Fri Dec 14 02:59:29.643 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] reconnect localhost:30002 failed couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] query failed to: localhost:30002 exception: socket exception [CONNECT_ERROR] for localhost:30002 m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30999| Fri Dec 14 02:59:29.643 [WriteBackListener-localhost:30000] WriteBackListener exception : all servers down/unreachable when querying: localhost:30000,localhost:30001,localhost:30002 m30999| Fri Dec 14 02:59:29.737 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:29.737 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.738 [Balancer] SyncClusterConnection connect fail to: localhost:30000 errmsg: couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.738 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:29.738 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.738 [Balancer] SyncClusterConnection connect fail to: localhost:30001 errmsg: couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.738 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:29.738 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.738 [Balancer] SyncClusterConnection connect fail to: localhost:30002 errmsg: couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.738 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.738 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.738 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.738 [Balancer] trying reconnect to localhost:30000 m30999| Fri Dec 14 02:59:29.738 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.738 [Balancer] reconnect localhost:30000 failed couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.738 [Balancer] trying reconnect to localhost:30001 m30999| Fri Dec 14 02:59:29.738 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.738 [Balancer] reconnect localhost:30001 failed couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.738 [Balancer] trying reconnect to localhost:30002 m30999| Fri Dec 14 02:59:29.739 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.739 [Balancer] reconnect localhost:30002 failed couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connecting to [localhost:30000] m30999| Fri Dec 14 02:59:29.739 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connect fail to: localhost:30000 errmsg: couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connecting to [localhost:30001] m30999| Fri Dec 14 02:59:29.739 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connect fail to: localhost:30001 errmsg: couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connecting to [localhost:30002] m30999| Fri Dec 14 02:59:29.739 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.739 [Balancer] SyncClusterConnection connect fail to: localhost:30002 errmsg: couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.739 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.739 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.739 [Balancer] unable to set SO_RCVTIMEO m30999| Fri Dec 14 02:59:29.739 [Balancer] trying reconnect to localhost:30000 m30999| Fri Dec 14 02:59:29.739 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.739 [Balancer] reconnect localhost:30000 failed couldn't connect to server localhost:30000 m30999| Fri Dec 14 02:59:29.740 [Balancer] query failed to: localhost:30000 exception: socket exception [CONNECT_ERROR] for localhost:30000 m30999| Fri Dec 14 02:59:29.740 [Balancer] trying reconnect to localhost:30001 m30999| Fri Dec 14 02:59:29.740 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.740 [Balancer] reconnect localhost:30001 failed couldn't connect to server localhost:30001 m30999| Fri Dec 14 02:59:29.740 [Balancer] query failed to: localhost:30001 exception: socket exception [CONNECT_ERROR] for localhost:30001 m30999| Fri Dec 14 02:59:29.740 [Balancer] trying reconnect to localhost:30002 m30999| Fri Dec 14 02:59:29.740 BackgroundJob starting: ConnectBG m30999| Fri Dec 14 02:59:29.740 [Balancer] reconnect localhost:30002 failed couldn't connect to server localhost:30002 m30999| Fri Dec 14 02:59:29.740 [Balancer] query failed to: localhost:30002 exception: socket exception [CONNECT_ERROR] for localhost:30002 m30999| Fri Dec 14 02:59:29.752 [Balancer] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30999| Fri Dec 14 02:59:29.752 [Balancer] scoped connection to localhost:30000,localhost:30001,localhost:30002 not being returned to the pool m30999| Fri Dec 14 02:59:29.752 [Balancer] caught exception while doing balance: all servers down/unreachable when querying: localhost:30000,localhost:30001,localhost:30002 m30999| Fri Dec 14 02:59:29.752 [Balancer] *** End of balancing round m30999| Fri Dec 14 02:59:30.617 [ Fri Dec 14 02:59:31.754 got signal 15 (Terminated), will terminate after current cmd ends Fri Dec 14 02:59:31.754 [interruptThread] now exiting Fri Dec 14 02:59:31.754 dbexit: Fri Dec 14 02:59:31.754 [interruptThread] shutdown: going to close listening sockets... Fri Dec 14 02:59:31.754 [interruptThread] closing listening socket: 11 Fri Dec 14 02:59:31.754 [interruptThread] closing listening socket: 12 Fri Dec 14 02:59:31.754 [interruptThread] closing listening socket: 13 Fri Dec 14 02:59:31.754 [interruptThread] removing socket file: /tmp/mongodb-27999.sock Fri Dec 14 02:59:31.754 [interruptThread] shutdown: going to flush diaglog... Fri Dec 14 02:59:31.754 [interruptThread] shutdown: going to close sockets... Fri Dec 14 02:59:31.754 [interruptThread] shutdown: waiting for fs preallocator... Fri Dec 14 02:59:31.754 [interruptThread] shutdown: closing all files... Fri Dec 14 02:59:31.756 [interruptThread] closeAllFiles() finished Fri Dec 14 02:59:31.756 [interruptThread] shutdown: removing fs lock... Fri Dec 14 02:59:31.760 dbexit: really exiting now