[SERVER-9338] db.runCommand( { compact: '<collection>' } ) on a primary decrements maintenanceMode Created: 11/Apr/13  Updated: 11/Jul/16  Resolved: 01/Jul/13

Status: Closed
Project: Core Server
Component/s: Replication, Shell
Affects Version/s: 2.2.3
Fix Version/s: 2.5.1

Type: Bug Priority: Major - P3
Reporter: Jenny Duckett Assignee: Andrew Emil (Inactive)
Resolution: Done Votes: 1
Labels: neweng, replicaset
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

GNU/Linux


Issue Links:
Duplicate
is duplicated by SERVER-9326 Node became SECONDARY in the middle o... Closed
Related
related to SERVER-9800 Maintenance mode commands should unse... Closed
is related to SERVER-6643 Maintenance mode should be unset when... Closed
Operating System: ALL
Steps To Reproduce:
  1. Create a replica set with 3 members under MongoDB 2.2.3
  2. Run db.runCommand( { compact: '<collection>' }

    ) in the mongo shell for the primary

  3. Note that this produces an error, as expected
  4. Run rs.status() - maintenanceMode will be -1 for the primary member.

Each time step 2 above is repeated, maintenanceMode is decremented one more for the primary (with or without force: true).

Participants:

 Description   

We have a replica set with 3 members running MongoDB 2.2.3. After running db.runCommand(

{ compact: '<collection>' }

) twice on the primary (first without force: true to see the expected error message, and then again with it to actually compact the collection), rs.status() showed maintenanceMode: -2 for the primary member. After some investigation, I ran the command again (without force: true) and confirmed that it decremented again to -3:

{
        "_id" : 0,
        "name" : "ec2-xxx.compute-1.amazonaws.com:27017",
        "health" : 1,
        "state" : 1,
        "stateStr" : "PRIMARY",
        "uptime" : 2833099,
        "optime" : Timestamp(1365604059000, 1),
        "optimeDate" : ISODate("2013-04-10T14:27:39Z"),
        "maintenanceMode" : -3,
        "self" : true
},

After stepping down the primary, maintenanceMode was not shown in rs.status() on the new primary, but on stepping that one down and the original primary taking over again, it was still -3 there. Restarting mongod appears to have reset maintenanceMode now and it no longer appears in rs.status().

I couldn't find any mention of this in the documentation or anywhere online. I'm not familiar with C++ so could easily have missed something, but it looks to me as if the problem might be that there is no check on theReplSet->isSecondary() when decrementing maintenanceMode in Command::execCommand, to correspond with that check when incrementing it earlier in the method. I have created a commit with this possible fix:

https://github.com/jennyd/mongo/commit/83a4b60e72d67bde124fed8072994da40945f297



 Comments   
Comment by auto [ 06/Jun/13 ]

Author:

{u'username': u'AndrewCEmil', u'name': u'AndrewCEmil', u'email': u'andrew.emil@10gen.com'}

Message: SERVER-9338 ensure maintenanceMode properly decremented
-created MaintenanceModeSetter for setting maintenance mode
-used MaintenanceModeSetter to avoid incorrectly decrementing maintenance mode
-added assert in rs.cpp to confirm maintenance mode never goes negative
Branch: master
https://github.com/mongodb/mongo/commit/80cc5892d74c9377d4a2c08087affe5b9852d120

Comment by aleksey [ 25/Apr/13 ]

Can we expect this fix to be backported to 2.4.x? It would be really helpful...

Generated at Thu Feb 08 03:20:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.