Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-51893

Ignore "can't connect to new replica set primary" in reconfig hook

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.0.0-rc0
    • Affects Version/s: None
    • Component/s: Replication
    • Labels:
      None
    • Fully Compatible
    • ALL
    • 17

      Quiesce mode throws ShutdownInProgress, so in SERVER-48492 we made the reconfig hook (run_reconfig_background.js) ignore ShutdownInProgress errors. However, we've still observed a failure, on commit ca65f33c on master, on this line in run_reconfig_background.js:

          const conn = connect(TestData.connectionString);
      

      The script logs:

      connecting to: mongodb://localhost:20500,localhost:20501,localhost:20502,localhost:20503,localhost:20504/?replicaSet=rs
      Error: can't connect to new replica set primary [localhost:20500], err: The server is in quiesce mode and will shut down :
      connect@src/mongo/shell/mongo.js:372:17
      @jstests/hooks/run_reconfig_background.js:121:18
      @jstests/hooks/run_reconfig_background.js:11:2
      failed to load: jstests/hooks/run_reconfig_background.js
      

      I hypothesize that the "connect" line threw an error with a code that is not ShutdownInProgress.

      Indeed, the client-side code that threw this error replaces the code with FailedToSatisfyReadPreference, in dbclient_rs.cpp:

              const std::string message = str::stream()
                  << "can't connect to new replica set primary [" << _primaryHost.toString() << "]"
                  << (errmsg.empty() ? "" : ", err: ") << errmsg;
              monitor->failedHost(_primaryHost, {ErrorCodes::Error(40659), message});
              uasserted(ErrorCodes::FailedToSatisfyReadPreference, message);
      

      I think the reconfig hook should ignore error messages like "can't connect to new replica set primary" in addition to its current error-suppression rules.

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: