Loading...

Type: Bug
Resolution: Done
Priority: Critical - P2
Fix Version/s: None
Affects Version/s: 2.6.7
Component/s: Index Maintenance
Labels:
None

Operating System:
ALL

Sharded cluster with replica sets (2 members 1 arbiter); after upgrading from 2.4.5 to 2.6.6 and/or 2.6.7 the mongod(s) crash at random times. Sometimes both members of a replica set crash one after the other..

2015-01-26T09:21:10.186-0500 [rsMgr] replSet info electSelf 0
2015-01-26T09:21:10.494-0500 [rsHealthPoll] replSet member machine4-lx:25920 is up
2015-01-26T09:21:10.494-0500 [rsHealthPoll] replSet member machine4-lx:25920 is now in state SECONDARY
2015-01-26T09:21:10.647-0500 [rsMgr] replSet PRIMARY
2015-01-26T09:21:10.650-0500 [rsMgr] CMD: drop ssp.tmp.mr.testresults_255_inc
2015-01-26T09:21:10.660-0500 [rsMgr] ERROR: About to fassert -  numIndexesTotal(): 0 numSystemIndexesEntries: 1 _entries.size(): 0 indexNamesToDrop: 1 haveIdIndex: 0
2015-01-26T09:21:10.660-0500 [rsMgr] ssp Fatal Assertion 17328
2015-01-26T09:21:10.679-0500 [rsMgr] ssp 0x11fd1b1 0x119efa9 0x1181add 0x8e5d0c 0x8c74cb 0xa3358e 0xa2949a 0xa2b611 0xa2cd36 0xd61654 0xba21a2 0xba3780 0xba4d06 0x7ebb1a 0x7b54ba 0xb95d66 0x7b87ca 0x7b8f22 0x7c5a40 0x7d1e2f 
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11fd1b1]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo10logContextEPKc+0x159) [0x119efa9]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo13fassertFailedEi+0xcd) [0x1181add]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12IndexCatalog14dropAllIndexesEb+0xf4c) [0x8e5d0c]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo8Database14dropCollectionERKNS_10StringDataE+0x37b) [0x8c74cb]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo7CmdDrop3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3ae) [0xa3358e]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2949a]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x1721) [0xa2b611]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c6) [0xa2cd36]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x2474) [0xd61654]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod() [0xba21a2]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x580) [0xba3780]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0xb6) [0xba4d06]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBClientCursor4initEv+0xba) [0x7ebb1a]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xea) [0x7b54ba]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x56) [0xb95d66]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo17DBClientInterface5findNERSt6vectorINS_7BSONObjESaIS2_EERKSsNS_5QueryEiiPKS2_i+0x9a) [0x7b87ca]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo17DBClientInterface7findOneERKSsRKNS_5QueryEPKNS_7BSONObjEi+0x72) [0x7b8f22]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo20DBClientWithCommands10runCommandERKSsRKNS_7BSONObjERS3_i+0xb0) [0x7c5a40]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo20DBClientWithCommands14dropCollectionERKSsPNS_7BSONObjE+0x15f) [0x7d1e2f]
2015-01-26T09:21:10.679-0500 [rsMgr] 

***aborting after fassert() failure


2015-01-26T09:21:10.695-0500 [rsMgr] SEVERE: Got signal: 6 (Aborted).
Backtrace:0x11fd1b1 0x11fc58e 0x3aee432920 0x3aee4328a5 0x3aee434085 0x1181b4a 0x8e5d0c 0x8c74cb 0xa3358e 0xa2949a 0xa2b611 0xa2cd36 0xd61654 0xba21a2 0xba3780 0xba4d06 0x7ebb1a 0x7b54ba 0xb95d66 0x7b87ca 
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11fd1b1]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod() [0x11fc58e]
 /lib64/libc.so.6() [0x3aee432920]
 /lib64/libc.so.6(gsignal+0x35) [0x3aee4328a5]
 /lib64/libc.so.6(abort+0x175) [0x3aee434085]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo13fassertFailedEi+0x13a) [0x1181b4a]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12IndexCatalog14dropAllIndexesEb+0xf4c) [0x8e5d0c]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo8Database14dropCollectionERKNS_10StringDataE+0x37b) [0x8c74cb]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo7CmdDrop3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3ae) [0xa3358e]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2949a]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x1721) [0xa2b611]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c6) [0xa2cd36]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x2474) [0xd61654]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod() [0xba21a2]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x580) [0xba3780]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0xb6) [0xba4d06]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBClientCursor4initEv+0xba) [0x7ebb1a]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xea) [0x7b54ba]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x56) [0xb95d66]
 /ssp/data1/mongodb/mongodb-linux-x86_64-2.6.7/bin/mongod(_ZN5mongo17DBClientInterface5findNERSt6vectorINS_7BSONObjESaIS2_EERKSsNS_5QueryEiiPKS2_i+0x9a) [0x7b87ca]

Details

Description

Attachments

Activity

People

Dates