[SERVER-51893] Ignore "can't connect to new replica set primary" in reconfig hook Created: 30/Oct/20  Updated: 29/Oct/23  Resolved: 02/Nov/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 5.0.0-rc0

Type: Bug Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: A. Jesse Jiryu Davis
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-48492 run_reconfig_background.js should ign... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:
Linked BF Score: 17

 Description   

Quiesce mode throws ShutdownInProgress, so in SERVER-48492 we made the reconfig hook (run_reconfig_background.js) ignore ShutdownInProgress errors. However, we've still observed a failure, on commit ca65f33c on master, on this line in run_reconfig_background.js:

    const conn = connect(TestData.connectionString);

The script logs:

connecting to: mongodb://localhost:20500,localhost:20501,localhost:20502,localhost:20503,localhost:20504/?replicaSet=rs
Error: can't connect to new replica set primary [localhost:20500], err: The server is in quiesce mode and will shut down :
connect@src/mongo/shell/mongo.js:372:17
@jstests/hooks/run_reconfig_background.js:121:18
@jstests/hooks/run_reconfig_background.js:11:2
failed to load: jstests/hooks/run_reconfig_background.js

I hypothesize that the "connect" line threw an error with a code that is not ShutdownInProgress.

Indeed, the client-side code that threw this error replaces the code with FailedToSatisfyReadPreference, in dbclient_rs.cpp:

        const std::string message = str::stream()
            << "can't connect to new replica set primary [" << _primaryHost.toString() << "]"
            << (errmsg.empty() ? "" : ", err: ") << errmsg;
        monitor->failedHost(_primaryHost, {ErrorCodes::Error(40659), message});
        uasserted(ErrorCodes::FailedToSatisfyReadPreference, message);

I think the reconfig hook should ignore error messages like "can't connect to new replica set primary" in addition to its current error-suppression rules.



 Comments   
Comment by Githook User [ 02/Nov/20 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}

Message: SERVER-51893 Ignore "can't connect to new replica set primary" in reconfig hook
Branch: master
https://github.com/mongodb/mongo/commit/2c8c742c516cbca9a8724eda610d3b01b0b3d62c

Generated at Thu Feb 08 05:26:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.