[SERVER-4350] Segmentation fault on replica recovery Created: 22/Nov/11  Updated: 11/Jul/16  Resolved: 23/Nov/11

Status: Closed
Project: Core Server
Component/s: Replication, Stability
Affects Version/s: 2.0.1, 2.0.2
Fix Version/s: 2.0.2, 2.1.0

Type: Bug Priority: Major - P3
Reporter: Catalin Constantin Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: core, dump, replication, sync
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Debian 6.0, 64bit, 8GB of RAM. Installed via the repository.


Issue Links:
Depends
is depended on by SERVER-4441 Got Signal: 11 (Segmentation Fault) u... Closed
Operating System: Linux
Participants:

 Description   

After cleaning /var/lib/mongodb and starting the replica server, sync starts.

Sync ends in core dump:

ue Nov 22 20:37:55 [rsSync] replSet syncing to: monster:27017
Tue Nov 22 20:37:55 [rsSync] build index newsman.bounce

{ _id: 1 }

Tue Nov 22 20:37:55 [rsSync] build index done 0 records 0.035 secs
Tue Nov 22 20:37:55 Invalid access at address: 0xbc

Tue Nov 22 20:37:55 Got signal: 11 (Segmentation fault).

Tue Nov 22 20:37:55 Backtrace:
0xa8d139 0xa8d710 0x7f9bd380df60 0x87d4d1 0x833c87 0x822bcb 0x825e9c 0x8271b5 0x82720a 0x827690 0xaa7b80 0x7f9bd38058ba 0x7f9bd2dc102d
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0xa8d139]
/usr/bin/mongod(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x220) [0xa8d710]
/lib/libpthread.so.0(+0xef60) [0x7f9bd380df60]
/usr/bin/mongod(_ZN5mongo7Helpers8findByIdEPNS_16NamespaceDetailsENS_7BSONObjE+0x21) [0x87d4d1]
/usr/bin/mongod(_ZN5mongo21applyOperation_inlockERKNS_7BSONObjEb+0xf27) [0x833c87]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl9syncApplyERKNS_7BSONObjE+0x1fb) [0x822bcb]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl8syncTailEv+0xd0c) [0x825e9c]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0xc5) [0x8271b5]
/usr/bin/mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x4a) [0x82720a]
/usr/bin/mongod(_ZN5mongo15startSyncThreadEv+0xa0) [0x827690]
/usr/bin/mongod(thread_proxy+0x80) [0xaa7b80]
/lib/libpthread.so.0(+0x68ba) [0x7f9bd38058ba]
/lib/libc.so.6(clone+0x6d) [0x7f9bd2dc102d]

Logstream::get called in uninitialized state
Tue Nov 22 20:37:55 ERROR: Client::~Client _context should be null but is not; client:rsSync
Logstream::get called in uninitialized state
Tue Nov 22 20:37:55 ERROR: Client::shutdown not called: rsSync



 Comments   
Comment by Eliot Horowitz (Inactive) [ 23/Nov/11 ]

Thanks - found the segfault.

Comment by auto [ 23/Nov/11 ]

Author:

{u'login': u'erh', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: fix oplog replay segfault SERVER-4350
Branch: v2.0
https://github.com/mongodb/mongo/commit/bd438596e1f9808663af9c476f0a08940b19df35

Comment by auto [ 23/Nov/11 ]

Author:

{u'login': u'erh', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: fix oplog replay segfault SERVER-4350
Branch: master
https://github.com/mongodb/mongo/commit/7156346ff727362dda1890e97c071eea338ed7c5

Comment by Catalin Constantin [ 22/Nov/11 ]

Rolling back to 2.0.0 solves the problem.

Comment by Catalin Constantin [ 22/Nov/11 ]

Also happens on 2.0.1:

Tue Nov 22 18:01:28 [initandlisten] MongoDB starting : pid=2422 port=27017 dbpath=/var/lib/mongodb 64-bit host=zeus
Tue Nov 22 18:01:28 [initandlisten] db version v2.0.1, pdfile version 4.5
Tue Nov 22 18:01:28 [initandlisten] git version: 3a5cf0e2134a830d38d2d1aae7e88ca31bdd684
Tue Nov 22 18:01:28 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41

Comment by Catalin Constantin [ 22/Nov/11 ]

Tue Nov 22 20:33:23 [initandlisten] MongoDB starting : pid=20956 port=27017 dbpath=/var/lib/mongodb 64-bit host=zeus
Tue Nov 22 20:33:23 [initandlisten] db version v2.0.2-rc1, pdfile version 4.5
Tue Nov 22 20:33:23 [initandlisten] git version: a91919f7261a24eb43fbe7f9635acabb0b2d157b
Tue Nov 22 20:33:23 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41

Comment by Kristina Chodorow (Inactive) [ 22/Nov/11 ]

Can you look at the beginning of the mongodb.log file and paste the version (the first couple of log lines)?

Comment by Catalin Constantin [ 22/Nov/11 ]

Data directory was empty.
I just got the error from mongodb.log file.

Comment by Eliot Horowitz (Inactive) [ 22/Nov/11 ]

Also - what version was this stack trace with?

Comment by Eliot Horowitz (Inactive) [ 22/Nov/11 ]

The data directory was completely empty when this started?

Generated at Thu Feb 08 03:05:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.