jstests/sharding/remove1.js failed on Linux 64 dur off

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Duplicate
    • Priority: Major - P3
    • None
    • Affects Version/s: 2.1.1
    • Component/s: None
    • None
    • ALL
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      May be sporadic. Need to understand the meaning of the failure mode.

      http://buildbot.mongodb.org/builders/Linux%2064-bit%20debug%20dur%20off/builds/1024/steps/test_9/logs/stdio

      Relevant section:

              Test : remove1.js ...
            Command : /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo --port 27999 --nodb /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/jstests/sharding/remove1.js --eval TestData = new Object();TestData.testPath = "/home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/jstests/sharding/remove1.js";TestData.testFile = "remove1.js";TestData.testName = "remove1";TestData.noJournal = false;TestData.noJournalPrealloc = false;TestData.auth = false;TestData.keyFile = null;TestData.keyFileData = null;
      MongoDB shell version: 2.1.1-pre-
      null
      Resetting db path '/data/db/remove_shard10'
      shell: started program /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod --port 30000 --dbpath /data/db/remove_shard10
       m30000| db level locking enabled: 0
       m30000| Fri Mar 30 12:13:43 [initandlisten] MongoDB starting : pid=11745 port=30000 dbpath=/data/db/remove_shard10 64-bit host=tp1.10gen.cc
       m30000| Fri Mar 30 12:13:43 [initandlisten] _DEBUG build (which is slower)
       m30000| Fri Mar 30 12:13:43 [initandlisten] 
       m30000| Fri Mar 30 12:13:43 [initandlisten] ** NOTE: This is a development version (2.1.1-pre-) of MongoDB.
       m30000| Fri Mar 30 12:13:43 [initandlisten] **       Not recommended for production.
       m30000| Fri Mar 30 12:13:43 [initandlisten] 
       m30000| Fri Mar 30 12:13:43 [initandlisten] db version v2.1.1-pre-, pdfile version 4.5
       m30000| Fri Mar 30 12:13:43 [initandlisten] git version: 5455f1d95c232c92647e3e0cfda4f52b28a08ef7
       m30000| Fri Mar 30 12:13:43 [initandlisten] build info: Linux tp1.10gen.cc 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009 x86_64 BOOST_LIB_VERSION=1_49
       m30000| Fri Mar 30 12:13:43 [initandlisten] options: { dbpath: "/data/db/remove_shard10", port: 30000 }
       m30000| Fri Mar 30 12:13:43 [initandlisten] opening db:  local
       m30000| Fri Mar 30 12:13:43 [websvr] admin web console waiting for connections on port 31000
       m30000| Fri Mar 30 12:13:43 [initandlisten] waiting for connections on port 30000
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53019 #1 (1 connection now open)
      Resetting db path '/data/db/remove_shard11'
      shell: started program /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod --port 30001 --dbpath /data/db/remove_shard11
       m30001| db level locking enabled: 0
       m30001| Fri Mar 30 12:13:43 [initandlisten] MongoDB starting : pid=11757 port=30001 dbpath=/data/db/remove_shard11 64-bit host=tp1.10gen.cc
       m30001| Fri Mar 30 12:13:43 [initandlisten] _DEBUG build (which is slower)
       m30001| Fri Mar 30 12:13:43 [initandlisten] 
       m30001| Fri Mar 30 12:13:43 [initandlisten] ** NOTE: This is a development version (2.1.1-pre-) of MongoDB.
       m30001| Fri Mar 30 12:13:43 [initandlisten] **       Not recommended for production.
       m30001| Fri Mar 30 12:13:43 [initandlisten] 
       m30001| Fri Mar 30 12:13:43 [initandlisten] db version v2.1.1-pre-, pdfile version 4.5
       m30001| Fri Mar 30 12:13:43 [initandlisten] git version: 5455f1d95c232c92647e3e0cfda4f52b28a08ef7
       m30001| Fri Mar 30 12:13:43 [initandlisten] build info: Linux tp1.10gen.cc 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009 x86_64 BOOST_LIB_VERSION=1_49
       m30001| Fri Mar 30 12:13:43 [initandlisten] options: { dbpath: "/data/db/remove_shard11", port: 30001 }
       m30001| Fri Mar 30 12:13:43 [initandlisten] opening db:  local
       m30001| Fri Mar 30 12:13:43 [websvr] admin web console waiting for connections on port 31001
       m30001| Fri Mar 30 12:13:43 [initandlisten] waiting for connections on port 30001
       m30001| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:47497 #1 (1 connection now open)
      "localhost:30000"
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53022 #2 (2 connections now open)
      ShardingTest remove_shard1 :
      {
      	"config" : "localhost:30000",
      	"shards" : [
      		connection to localhost:30000,
      		connection to localhost:30001
      	]
      }
       m30000| Fri Mar 30 12:13:43 [conn2] opening db:  config
      shell: started program /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos --port 30999 --configdb localhost:30000
       m30999| Fri Mar 30 12:13:43 warning: running with 1 config server should be done only for testing purposes and is not recommended for production
       m30999| Fri Mar 30 12:13:43 [mongosMain] MongoS version 2.1.1-pre- starting: pid=11771 port=30999 64-bit host=tp1.10gen.cc (--help for usage)
       m30999| Fri Mar 30 12:13:43 [mongosMain] _DEBUG build
       m30999| Fri Mar 30 12:13:43 [mongosMain] git version: 5455f1d95c232c92647e3e0cfda4f52b28a08ef7
       m30999| Fri Mar 30 12:13:43 [mongosMain] build info: Linux tp1.10gen.cc 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009 x86_64 BOOST_LIB_VERSION=1_49
       m30999| Fri Mar 30 12:13:43 [mongosMain] options: { configdb: "localhost:30000", port: 30999 }
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53023 #3 (3 connections now open)
       m30000| Fri Mar 30 12:13:43 [FileAllocator] allocating new datafile /data/db/remove_shard10/config.ns, filling with zeroes...
       m30000| Fri Mar 30 12:13:43 [FileAllocator] creating directory /data/db/remove_shard10/_tmp
       m30000| Fri Mar 30 12:13:43 [FileAllocator] done allocating datafile /data/db/remove_shard10/config.ns, size: 16MB,  took 0.016 secs
       m30000| Fri Mar 30 12:13:43 [FileAllocator] allocating new datafile /data/db/remove_shard10/config.0, filling with zeroes...
       m30000| Fri Mar 30 12:13:43 [FileAllocator] done allocating datafile /data/db/remove_shard10/config.0, size: 64MB,  took 0.018 secs
       m30000| Fri Mar 30 12:13:43 [conn2] datafileheader::init initializing /data/db/remove_shard10/config.0 n:0
       m30000| Fri Mar 30 12:13:43 [FileAllocator] allocating new datafile /data/db/remove_shard10/config.1, filling with zeroes...
       m30000| Fri Mar 30 12:13:43 [conn2] _DEBUG info _PAGEFAULTEXCEPTION is off
       m30000| Fri Mar 30 12:13:43 [conn2] build index config.settings { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn2] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53025 #4 (4 connections now open)
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53026 #5 (5 connections now open)
       m30000| Fri Mar 30 12:13:43 [conn5] build index config.version { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn5] build index done.  scanned 0 total records. 0 secs
       m30999| Fri Mar 30 12:13:43 [websvr] admin web console waiting for connections on port 31999
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.chunks { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] info: creating collection config.chunks on add index
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.chunks { ns: 1, min: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.chunks { ns: 1, shard: 1, min: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.chunks { ns: 1, lastmod: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.shards { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] info: creating collection config.shards on add index
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.shards { host: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30999| Fri Mar 30 12:13:43 [Balancer] about to contact config servers and shards
       m30999| Fri Mar 30 12:13:43 [mongosMain] waiting for connections on port 30999
       m30999| Fri Mar 30 12:13:43 [Balancer] config servers and shards contacted successfully
       m30999| Fri Mar 30 12:13:43 [Balancer] balancer id: tp1.10gen.cc:30999 started at Mar 30 12:13:43
       m30999| Fri Mar 30 12:13:43 [Balancer] created new distributed lock for balancer on localhost:30000 ( lock timeout : 900000, ping interval : 30000, process : 0 )
       m30000| Fri Mar 30 12:13:43 [conn5] build index config.mongos { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn5] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53027 #6 (6 connections now open)
       m30000| Fri Mar 30 12:13:43 [FileAllocator] done allocating datafile /data/db/remove_shard10/config.1, size: 128MB,  took 0.017 secs
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.lockpings { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.lockpings { ping: 1 }
       m30999| Fri Mar 30 12:13:43 [LockPinger] creating distributed lock ping thread for localhost:30000 and process tp1.10gen.cc:30999:1333127623:1804289383 (sleeping for 30000ms)
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 1 total records. 0 secs
       m30000| Fri Mar 30 12:13:43 [conn6] build index config.locks { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn6] build index done.  scanned 0 total records. 0 secs
       m30999| Fri Mar 30 12:13:43 [Balancer] distributed lock 'balancer/tp1.10gen.cc:30999:1333127623:1804289383' acquired, ts : 4f75e9c7cb03628d97f227c9
       m30999| Fri Mar 30 12:13:43 [Balancer] distributed lock 'balancer/tp1.10gen.cc:30999:1333127623:1804289383' unlocked. 
       m30999| Fri Mar 30 12:13:43 [mongosMain] connection accepted from 127.0.0.1:41026 #1 (1 connection now open)
      ShardingTest undefined going to add shard : localhost:30000
       m30999| Fri Mar 30 12:13:43 [conn1] couldn't find database [admin] in config db
       m30000| Fri Mar 30 12:13:43 [conn4] build index config.databases { _id: 1 }
       m30000| Fri Mar 30 12:13:43 [conn4] build index done.  scanned 0 total records. 0 secs
       m30999| Fri Mar 30 12:13:43 [conn1] 	 put [admin] on: config:localhost:30000
       m30999| Fri Mar 30 12:13:43 [conn1] going to add shard: { _id: "shard0000", host: "localhost:30000" }
      { "shardAdded" : "shard0000", "ok" : 1 }
      ShardingTest undefined going to add shard : localhost:30001
       m30001| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:47505 #2 (2 connections now open)
       m30999| Fri Mar 30 12:13:43 [conn1] going to add shard: { _id: "shard0001", host: "localhost:30001" }
      { "shardAdded" : "shard0001", "ok" : 1 }
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53030 #7 (7 connections now open)
       m30999| Fri Mar 30 12:13:43 [conn1] creating WriteBackListener for: localhost:30000 serverID: 4f75e9c7cb03628d97f227c8
       m30001| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:47507 #3 (3 connections now open)
       m30999| Fri Mar 30 12:13:43 [conn1] creating WriteBackListener for: localhost:30001 serverID: 4f75e9c7cb03628d97f227c8
       m30999| Fri Mar 30 12:13:43 [conn1] going to start draining shard: shard0000
       m30999| primaryLocalDoc: { _id: "local", primary: "shard0000" }
       m30000| Fri Mar 30 12:13:43 [initandlisten] connection accepted from 127.0.0.1:53032 #8 (8 connections now open)
       m30999| Fri Mar 30 12:13:43 [conn1] DROP DATABASE: needToMove
       m30999| Fri Mar 30 12:13:43 [conn1] DBConfig::dropDatabase: needToMove
       m30999| Fri Mar 30 12:13:43 [conn1] about to log metadata event: { _id: "tp1.10gen.cc-2012-03-30T17:13:43-0", server: "tp1.10gen.cc", clientAddr: "N/A", time: new Date(1333127623616), what: "dropDatabase.start", ns: "needToMove", details: {} }
       m30999| Fri Mar 30 12:13:43 [conn1]    DBConfig::dropDatabase: needToMove dropped sharded collections: 0
       m30000| Fri Mar 30 12:13:43 [conn4] opening db:  needToMove
       m30000| Fri Mar 30 12:13:43 [conn4] dropDatabase needToMove
       m30999| Fri Mar 30 12:13:43 [conn1] about to log metadata event: { _id: "tp1.10gen.cc-2012-03-30T17:13:43-1", server: "tp1.10gen.cc", clientAddr: "N/A", time: new Date(1333127623619), what: "dropDatabase", ns: "needToMove", details: {} }
       m30999| Fri Mar 30 12:13:43 [conn1] going to remove shard: shard0000
      assert: supposed to be null (should have removed local db) was: { "_id" : "local", "partitioned" : false, "primary" : "shard0000" }
      Error("Printing Stack Trace")@:0
      ()@src/mongo/shell/utils.js:37
      ("supposed to be null (should have removed local db) was: { \"_id\" : \"local\", \"partitioned\" : false, \"primary\" : \"shard0000\" }")@src/mongo/shell/utils.js:58
      ([object bson_object],"should have removed local db")@src/mongo/shell/utils.js:235
      @/home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/jstests/sharding/remove1.js:18
      
      Fri Mar 30 12:13:43 uncaught exception: supposed to be null (should have removed local db) was: { "_id" : "local", "partitioned" : false, "primary" : "shard0000" }
      failed to load: /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/jstests/sharding/remove1.js
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] dev: lastError==0 won't report:DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('4f75e9c7cb03628d97f227c8') }
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30000 ns: admin.$cmd query: { writebacklisten: ObjectId('4f75e9c7cb03628d97f227c8') }
       m30000| Fri Mar 30 12:13:43 got signal 15 (Terminated), will terminate after current cmd ends
       m30000| Fri Mar 30 12:13:43 [interruptThread] now exiting
       m30000| Fri Mar 30 12:13:43 dbexit: 
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: going to close listening sockets...
       m30000| Fri Mar 30 12:13:43 [interruptThread] closing listening socket: 35
       m30000| Fri Mar 30 12:13:43 [interruptThread] closing listening socket: 36
       m30000| Fri Mar 30 12:13:43 [interruptThread] closing listening socket: 37
       m30000| Fri Mar 30 12:13:43 [interruptThread] removing socket file: /tmp/mongodb-30000.sock
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: going to flush diaglog...
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: going to close sockets...
       m30000| Fri Mar 30 12:13:43 [conn6] end connection 127.0.0.1:53027 (7 connections now open)
       m30000| Fri Mar 30 12:13:43 [conn6] thread conn stack usage was 10176 bytes
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] Socket recv() errno:104 Connection reset by peer 127.0.0.1:30000
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] SocketException: remote: 127.0.0.1:30000 error: 9001 socket exception [1] server [127.0.0.1:30000] 
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] DBClientCursor::init call() failed
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] Assertion: 13632:couldn't get updated shard list from config server
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] dev: lastError==0 won't report:couldn't get updated shard list from config server
       m30999| 0x794b6f 0x9f83fb 0xb10700 0xb0f377 0x7b2009 0x87d0f9 0x8801b7 0x8800b7 0x880035 0x87ffc2 0x83b1f4 0x7f303193ea04 0x7f3030efcd4d 
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo15printStackTraceERSo+0x27) [0x794b6f]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo11msgassertedEiPKc+0xfd) [0x9f83fb]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo15StaticShardInfo6reloadEv+0x15c) [0xb10700]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo5Shard15reloadShardInfoEv+0x13) [0xb0f377]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo17WriteBackListener3runEv+0x1be9) [0x7b2009]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x35d) [0x87d0f9]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZNK5boost4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS3_9JobStatusEEEEclEPS3_S6_+0x7d) [0x8801b7]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost3_bi5list2INS0_5valueIPN5mongo13BackgroundJobEEENS2_INS_10shared_ptrINS4_9JobStatusEEEEEEclINS_4_mfi3mf1IvS4_S9_EENS0_5list0EEEvNS0_4typeIvEERT_RT0_i+0x79) [0x8800b7]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS5_9JobStatusEEEEENS0_5list2INS0_5valueIPS5_EENSB_IS8_EEEEEclEv+0x3f) [0x880035]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS7_9JobStatusEEEEENS2_5list2INS2_5valueIPS7_EENSD_ISA_EEEEEEE3runEv+0x1e) [0x87ffc2]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos [0x83b1f4]
       m30999|  /lib/libpthread.so.0 [0x7f303193ea04]
       m30999|  /lib/libc.so.6(clone+0x6d) [0x7f3030efcd4d]
       m30999| Fri Mar 30 12:13:43 [WriteBackListener-localhost:30000] ERROR: backgroundjob WriteBackListener-localhost:30000error: couldn't get updated shard list from config server
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: waiting for fs preallocator...
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: closing all files...
       m30000| Fri Mar 30 12:13:43 [interruptThread] closeAllFiles() finished
       m30000| Fri Mar 30 12:13:43 [interruptThread] shutdown: removing fs lock...
       m30000| Fri Mar 30 12:13:43 dbexit: really exiting now
       m30001| Fri Mar 30 12:13:44 got signal 15 (Terminated), will terminate after current cmd ends
       m30001| Fri Mar 30 12:13:44 [interruptThread] now exiting
       m30001| Fri Mar 30 12:13:44 dbexit: 
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: going to close listening sockets...
       m30001| Fri Mar 30 12:13:44 [interruptThread] closing listening socket: 38
       m30001| Fri Mar 30 12:13:44 [interruptThread] closing listening socket: 39
       m30001| Fri Mar 30 12:13:44 [interruptThread] closing listening socket: 40
       m30001| Fri Mar 30 12:13:44 [interruptThread] removing socket file: /tmp/mongodb-30001.sock
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: going to flush diaglog...
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: going to close sockets...
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: waiting for fs preallocator...
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: closing all files...
       m30001| Fri Mar 30 12:13:44 [interruptThread] closeAllFiles() finished
       m30001| Fri Mar 30 12:13:44 [interruptThread] shutdown: removing fs lock...
       m30001| Fri Mar 30 12:13:44 dbexit: really exiting now
       m30001| Fri Mar 30 12:13:44 [interruptThread] thread interruptThread stack usage was 4560 bytes
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] dev: lastError==0 won't report:DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('4f75e9c7cb03628d97f227c8') }
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] WriteBackListener exception : DBClientBase::findN: transport error: localhost:30001 ns: admin.$cmd query: { writebacklisten: ObjectId('4f75e9c7cb03628d97f227c8') }
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] DBClientCursor::init call() failed
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] Assertion: 13632:couldn't get updated shard list from config server
       m30999| 0x794b6f 0x9f83fb 0xb10700 0xb0f377 0x7b2009 0x87d0f9 0x8801b7 0x8800b7 0x880035 0x87ffc2 0x83b1f4 0x7f303193ea04 0x7f3030efcd4d 
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo15printStackTraceERSo+0x27) [0x794b6f]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo11msgassertedEiPKc+0xfd) [0x9f83fb]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo15StaticShardInfo6reloadEv+0x15c) [0xb10700]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo5Shard15reloadShardInfoEv+0x13) [0xb0f377]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo17WriteBackListener3runEv+0x1be9) [0x7b2009]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x35d) [0x87d0f9]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZNK5boost4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS3_9JobStatusEEEEclEPS3_S6_+0x7d) [0x8801b7]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost3_bi5list2INS0_5valueIPN5mongo13BackgroundJobEEENS2_INS_10shared_ptrINS4_9JobStatusEEEEEEclINS_4_mfi3mf1IvS4_S9_EENS0_5list0EEEvNS0_4typeIvEERT_RT0_i+0x79) [0x8800b7]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS5_9JobStatusEEEEENS0_5list2INS0_5valueIPS5_EENSB_IS8_EEEEEclEv+0x3f) [0x880035]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS7_9JobStatusEEEEENS2_5list2INS2_5valueIPS7_EENSD_ISA_EEEEEEE3runEv+0x1e) [0x87ffc2]
       m30999|  /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongos [0x83b1f4]
       m30999|  /lib/libpthread.so.0 [0x7f303193ea04]
       m30999|  /lib/libc.so.6(clone+0x6d) [0x7f3030efcd4d]
       m30999| Fri Mar 30 12:13:44 [WriteBackListener-localhost:30001] ERROR: backgroundjob WriteBackListener-localhost:30001error: couldn't get updated shard list from config server
       m30999| Fri Mar 30 12:13:45 [mongosMain] dbexit: received signal 15 rc:0 received signal 15
       m30999| _DEBUG warning trying to lock a mongo::mutex during static shutdown
       m30999| _DEBUG warning trying to lock a mongo::mutex during static shutdown
                      3796.560049ms
      Fri Mar 30 12:13:46 got signal 15 (Terminated), will terminate after current cmd ends
      Fri Mar 30 12:13:46 [interruptThread] now exiting
      Fri Mar 30 12:13:46 dbexit: 
      Fri Mar 30 12:13:46 [interruptThread] shutdown: going to close listening sockets...
      Fri Mar 30 12:13:46 [interruptThread] closing listening socket: 5
      Fri Mar 30 12:13:46 [interruptThread] closing listening socket: 6
      Fri Mar 30 12:13:46 [interruptThread] closing listening socket: 7
      Fri Mar 30 12:13:46 [interruptThread] removing socket file: /tmp/mongodb-27999.sock
      Fri Mar 30 12:13:46 [interruptThread] shutdown: going to flush diaglog...
      Fri Mar 30 12:13:46 [interruptThread] shutdown: going to close sockets...
      Fri Mar 30 12:13:46 [interruptThread] shutdown: waiting for fs preallocator...
      Fri Mar 30 12:13:46 [interruptThread] shutdown: closing all files...
      Fri Mar 30 12:13:46 [interruptThread] closeAllFiles() finished
      Fri Mar 30 12:13:46 [interruptThread] shutdown: removing fs lock...
      Fri Mar 30 12:13:46 dbexit: really exiting now
      test /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/jstests/sharding/remove1.js exited with status 253
      

            Assignee:
            Eliot Horowitz (Inactive)
            Reporter:
            Andy Schwerin
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: