Mon Jan 14 16:11:34 [rsStart] replSet I am qa-2:27017
|
Mon Jan 14 16:11:34 [rsStart] replSet STARTUP2
|
Mon Jan 14 16:11:34 [rsMgr] replSet total number of votes is even - add arbiter or give one member an extra vote
|
Mon Jan 14 16:11:34 [rsHealthPoll] replSet member qa-1:27017 is up
|
Mon Jan 14 16:11:34 [rsHealthPoll] replSet member qa-1:27017 is now in state SECONDARY
|
Mon Jan 14 16:11:35 [rsSync] replSet SECONDARY
|
Mon Jan 14 16:11:35 [initandlisten] connection accepted from 10.0.1.26:47386 #13 (13 connections now open)
|
Mon Jan 14 16:11:35 [conn13] end connection 10.0.1.26:47386 (12 connections now open)
|
Mon Jan 14 16:11:35 [initandlisten] connection accepted from 10.0.1.26:47387 #14 (13 connections now open)
|
Mon Jan 14 16:11:38 [initandlisten] connection accepted from 10.0.5.101:33084 #15 (14 connections now open)
|
Mon Jan 14 16:11:38 [initandlisten] connection accepted from 10.0.5.101:33085 #16 (15 connections now open)
|
Mon Jan 14 16:11:38 [initandlisten] connection accepted from 10.0.5.101:33086 #17 (16 connections now open)
|
Mon Jan 14 16:11:41 [rsBackgroundSync] replSet syncing to: qa-1:27017
|
Mon Jan 14 16:11:41 [conn14] replSet info voting yea for qa-1:27017 (1)
|
Mon Jan 14 16:11:42 [rsSyncNotifier] replset setting oplog notifier to qa-1:27017
|
Mon Jan 14 16:11:42 [rsHealthPoll] replSet member qa-1:27017 is now in state PRIMARY
|
Mon Jan 14 16:11:42 [repl writer worker 2] btree bucket corrupt? consider reindexing or running validate command
|
Mon Jan 14 16:11:42 [repl writer worker 2] klast: { : { 54430: new Date(1316110703555) } }
|
Mon Jan 14 16:11:42 [repl writer worker 2] key: { : { 54433: new Date(1315512110121) } }
|
Mon Jan 14 16:11:42 [repl writer worker 2] ctp.thermostat Assertion failure false src/mongo/db/btree.cpp 359
|
0xaffd31 0xac65cd 0x62fd4b 0x636358 0x637377 0x791339 0x7876fe 0x79cea5 0x86cc47 0x86df23 0x84216b 0x844937 0x823598 0x9a34c2 0x9a2d4f 0xad35cd 0xb45ba9 0x7feba55819ca 0x7feba492821d
|
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
|
/usr/bin/mongod(_ZN5mongo12verifyFailedEPKcS1_j+0xfd) [0xac65cd]
|
/usr/bin/mongod(_ZN5mongo12BucketBasicsINS_12BtreeData_V0EE9_pushBackENS_7DiskLocERKNS_7KeyBsonERKNS_8OrderingES3_+0x21b) [0x62fd4b]
|
/usr/bin/mongod(_ZN5mongo11BtreeBucketINS_12BtreeData_V0EE5splitENS_7DiskLocEiS3_RKNS_7KeyBsonERKNS_8OrderingES3_S3_RNS_12IndexDetailsE+0x128) [0x636358]
|
/usr/bin/mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V0EE10insertHereENS_7DiskLocEiS3_RKNS_7KeyBsonERKNS_8OrderingES3_S3_RNS_12IndexDetailsE+0x947) [0x637377]
|
/usr/bin/mongod(_ZNK5mongo30IndexInsertionContinuationImplINS_12BtreeData_V0EE22doIndexInsertionWritesEv+0x89) [0x791339]
|
/usr/bin/mongod(_ZN5mongo14IndexInterface13IndexInserter19finishAllInsertionsEv+0x1e) [0x7876fe]
|
/usr/bin/mongod(_ZN5mongo24indexRecordUsingTwoStepsEPKcPNS_16NamespaceDetailsENS_7BSONObjENS_7DiskLocEb+0x4e5) [0x79cea5]
|
/usr/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibbPb+0xf87) [0x86cc47]
|
/usr/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb+0x43) [0x86df23]
|
/usr/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x322b) [0x84216b]
|
/usr/bin/mongod(_ZN5mongo27updateObjectsForReplicationEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x844937]
|
/usr/bin/mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEbb+0x1228) [0x823598]
|
/usr/bin/mongod(_ZN5mongo7replset8SyncTail9syncApplyERKNS_7BSONObjEb+0x292) [0x9a34c2]
|
/usr/bin/mongod(_ZN5mongo7replset14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x4f) [0x9a2d4f]
|
/usr/bin/mongod(_ZN5mongo10threadpool6Worker4loopEv+0x26d) [0xad35cd]
|
/usr/bin/mongod() [0xb45ba9]
|
/lib/libpthread.so.0(+0x69ca) [0x7feba55819ca]
|
/lib/libc.so.6(clone+0x6d) [0x7feba492821d]
|
Mon Jan 14 16:11:42 [repl writer worker 2] ERROR: writer worker caught exception: assertion src/mongo/db/btree.cpp:359 on: { ts: Timestamp 1358179394000|30, h: 7248757333306923217, v: 2, op: "i", ns: "ctp.thermostat", o: { _id: ObjectId('50f42c427258a6756200007c'), uuid: "a26c4db8c14b", organizationId: 17215, userIds: [ 54391 ], platform: "RTCOA_3M50", name: "fakestat" } }
|
Mon Jan 14 16:11:42 [repl writer worker 2] Fatal Assertion 16360
|
0xaffd31 0xac5323 0x9a2e56 0xad35cd 0xb45ba9 0x7feba55819ca 0x7feba492821d
|
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
|
/usr/bin/mongod(_ZN5mongo13fassertFailedEi+0xa3) [0xac5323]
|
/usr/bin/mongod(_ZN5mongo7replset14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x156) [0x9a2e56]
|
/usr/bin/mongod(_ZN5mongo10threadpool6Worker4loopEv+0x26d) [0xad35cd]
|
/usr/bin/mongod() [0xb45ba9]
|
/lib/libpthread.so.0(+0x69ca) [0x7feba55819ca]
|
/lib/libc.so.6(clone+0x6d) [0x7feba492821d]
|
Mon Jan 14 16:11:42 [repl writer worker 2]
|
|
***aborting after fassert() failure
|
|
|
Mon Jan 14 16:11:42 Got signal: 6 (Aborted).
|
|
Mon Jan 14 16:11:42 Backtrace:
|
0xaffd31 0x558bb9 0x7feba4872ba0 0x7feba4872b25 0x7feba4876670 0xac535e 0x9a2e56 0xad35cd 0xb45ba9 0x7feba55819ca 0x7feba492821d
|
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
|
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x558bb9]
|
/lib/libc.so.6(+0x33ba0) [0x7feba4872ba0]
|
/lib/libc.so.6(gsignal+0x35) [0x7feba4872b25]
|
/lib/libc.so.6(abort+0x180) [0x7feba4876670]
|
/usr/bin/mongod(_ZN5mongo13fassertFailedEi+0xde) [0xac535e]
|
/usr/bin/mongod(_ZN5mongo7replset14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x156) [0x9a2e56]
|
/usr/bin/mongod(_ZN5mongo10threadpool6Worker4loopEv+0x26d) [0xad35cd]
|
/usr/bin/mongod() [0xb45ba9]
|
/lib/libpthread.so.0(+0x69ca) [0x7feba55819ca]
|
/lib/libc.so.6(clone+0x6d) [0x7feba492821d]
|