[SERVER-5945] Reconfiguring replica set doesn't work sometimes Created: 28/May/12  Updated: 11/Jun/14  Resolved: 05/Sep/12

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.0.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Nick Demyanchuk Assignee: Scott Hernandez (Inactive)
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Centos 5.6 x86_64, EC2 m1.small instance,


Attachments: Text File mongodb.shardsrv.log    
Operating System: Linux
Participants:

 Description   

I was playing with reconfiguring replica set, when some nodes are down (using force argument).

Sometimes mongo answers with

{u'msg': u'will try this config momentarily, try running rs.conf() again in a few seconds', u'ok': 1.0}

, but
replica set configuration doesn't change. I can't find any sign of replSetReconfig equest in mongo log. I use latest pymongo driver.



 Comments   
Comment by Nick Demyanchuk [ 11/Jun/14 ]

It is indeed 2.6.1, nothing changed except version of mongo. I assume it's race condition somewhere in mongo code, since all those mongo commands were executed by script right after mongo was ready to receive commands (you can see it by timestamps).

I will try to reproduce it and let you know. Previous config contained mongo-0-0:27018 as primary, mongo-0-1:27018 as secondary and mongo-0-0:27020 as arbiter.

Comment by Scott Hernandez (Inactive) [ 10/Jun/14 ]

It seems like this is now 2.6.1, and no longer 2.0.x, is that correct?

If so I'd like to any new information which may have changed. Also, increased logging would help, if you can reproduce it.

Please include the old config before the reconfig as well.

Comment by Nick Demyanchuk [ 10/Jun/14 ]

It happened again, and i have logs now. You can see first time script tried to run replSetReconfig without force argument, but it failed. Then script retried with force argument, and received result:

{u'msg': u'will try this config momentarily, try running rs.conf() again in a few seconds', u'ok': 1.0}

but it didn't work - you can see that rsHealthPoll is still trying to reach members that should not be members of replica set already, and node never got PRIMARY state.

Comment by Ian Whalen (Inactive) [ 05/Sep/12 ]

nick, I'm closing this issue for lack if info but please reopen if you're able to attach the relevant logs to continue debugging.

Comment by Scott Hernandez (Inactive) [ 28/May/12 ]

Can you please attach the logs from the server you are issuing the command to? Please increase the logging level to 6 before running the command by using setParameter (http://www.mongodb.org/display/DOCS/setParameter+Command). You can issue the setParameter command before and after the replSetReconfig to turn up and then down logging.

Generated at Thu Feb 08 03:10:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.