[SERVER-17771] Reconfiguring a replica set to remove a node causes a segmentation fault on 2.6.8 Created: 27/Mar/15  Updated: 04/Nov/15  Resolved: 31/Mar/15

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.6.8
Fix Version/s: 2.6.10

Type: Bug Priority: Major - P3
Reporter: James O'Leary Assignee: Matt Dannenberg
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux, MongoDB 2.6.8


Issue Links:
Related
Backwards Compatibility: Fully Compatible
Operating System: Linux
Participants:

 Description   

The replSetReconfig command can generate the following Segmentation Fault and stack trace:

2015-03-27T13:46:53.462+0100 [SyncSourceFeedbackThread] SyncSourceFeedback error sending update, response: { ok: 0.0, errmsg: "could not update position upstream; will retry" }
2015-03-27T13:46:53.462+0100 [SyncSourceFeedbackThread] replset setting syncSourceFeedback to XXXX-XXXXX:27017
2015-03-27T13:46:53.464+0100 [SyncSourceFeedbackThread] sync source does not have member 1 in its config and neither do we, removing member from tracking
2015-03-27T13:46:53.464+0100 [SyncSourceFeedbackThread] sync source does not have member 1 in its config and neither do we, removing member from tracking
2015-03-27T13:46:53.467+0100 [SyncSourceFeedbackThread] SEVERE: Invalid access at address: 0x8
2015-03-27T13:46:53.494+0100 [SyncSourceFeedbackThread] SEVERE: Got signal: 11 (Segmentation fault).
Backtrace:0x1205431 0x120480e 0x12048ff 0x7f22c747f0a0 0xe8be4f 0xecdc3d 0xed2a73 0x118cf12 0x1249dc9 0x7f22c7476b50 0x7f22c681995d 
 ./bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x1205431]
 ./bin/mongod() [0x120480e]
 ./bin/mongod() [0x12048ff]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0) [0x7f22c747f0a0]
 ./bin/mongod(_ZNK5mongo13ReplSetConfig9MemberCfg6asBsonEv+0x3f) [0xe8be4f]
 ./bin/mongod(_ZN5mongo18SyncSourceFeedback14updateUpstreamEv+0x78d) [0xecdc3d]
 ./bin/mongod(_ZN5mongo18SyncSourceFeedback3runEv+0x8d3) [0xed2a73]
 ./bin/mongod(_ZN5mongo13BackgroundJob7jobBodyEv+0xd2) [0x118cf12]
 ./bin/mongod() [0x1249dc9]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x7f22c7476b50]
 /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f22c681995d]

The root cause appears to be due to removing a node while there are active writes.

The workaround for this issue is to shutdown the node to be removed before issuing the reconfiguration command.

At the moment this does not appear to be an issue for 2.8 and above.



 Comments   
Comment by Githook User [ 31/Mar/15 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-17771 do not track progress of recently removed nodes
Branch: v2.6
https://github.com/mongodb/mongo/commit/0b314f3157d987fd59194bc29dc780d9be367598

Generated at Thu Feb 08 03:45:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.