[SERVER-32679] shutdown command fails on single node replica set for timeoutSecs >= 120 Created: 12/Jan/18  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: James O'Leary Assignee: Backlog - Replication Team
Resolution: Unresolved Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-47190 Shutdown command with force:true shou... Closed
Assigned Teams:
Replication
Operating System: ALL
Participants:

 Description   

Running shutdownServer on a single node replica as follows fails:

$ mongo  --eval 'db.setLogLevel(1, "replication");db.getSiblingDB("admin").shutdownServer({"force": true, "timeoutSecs": 121})' ; echo "status = $?"
MongoDB shell version v3.7.0-472-g11225541cb
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.7.0-472-g11225541cb
2018-01-12T16:32:52.267+0000 E QUERY    [thread1] Error: shutdownServer failed: {
        "ok" : 0,
        "errmsg" : "By the time we were ready to step down, we were already past the time we were supposed to step down until",
        "code" : 50,
        "codeName" : "ExceededTimeLimit",
        "operationTime" : Timestamp(1515774634, 5),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1515774634, 5),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype.shutdownServer@src/mongo/shell/db.js:453:1
@(shell eval):1:34
status = 252

Running shutdownServer with any value less than this will succeed:

 
$ mongo  --eval 'db.setLogLevel(1, "replication");db.getSiblingDB("admin").shutdownServer({"force": true, "timeoutSecs": 119})' ; echo "status = $?"
MongoDB shell version v3.7.0-472-g11225541cb
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.7.0-472-g11225541cb
server should be down...
status = 0

The same behavior manifest on 3.4 / 3.6.



 Comments   
Comment by Lingzhi Deng [ 14/Apr/20 ]

We have SERVER-42525 which fixed the the stepDown in non-command shutdown path. But since we have two shutdown paths that run stepDown, we still try to stepDown for single node replset in the command shutdown path.

Comment by Siyuan Zhou [ 09/Apr/20 ]

There are two problems in this issue.

  1. I believe we should skip stepdown on single node replset shutdown, since there's no secondary that can take over. For single voter replset, shutdown should error immediately.
  2. Force shutdown should not wait for anything to step down, it's unclear why shutdown waits.
  3. The stepdown on shutdown code may break an assumption that stepdown period must be longer than secondaryCatchUpPeriodSecs which is checked by stepdown command.
Generated at Thu Feb 08 04:30:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.