• Type: Bug
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Catalog and Routing
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      When a replica set member has members[].priorityPort set (and disableReplicationUsageOfPriorityPort is false), replSetGetStatus always reports members[].pingMs: 0 for that member. The heartbeat RTT is recorded, but under a different key than the one replSetGetStatus reads.

      Cause

      • Heartbeats are sent to getHostAndPortPriority() (replication_coordinator_impl_heartbeat.cpp, auto target = rsc.getMemberAt.getHostAndPortPriority();). The ping stats are therefore recorded under the priority host:port: PingStats& hbStats = _pings[target]; in TopologyCoordinator::prepareHeartbeatRequestV1 and TopologyCoordinator::processHeartbeatResponse.
      • TopologyCoordinator::prepareStatusResponse looks the ping up by the main host:port: Milliseconds ping = _getPing(itConfig.getHostAndPort()); (topology_coordinator.cpp:2325 on master at 73ba7296d6d; :2235 on v9.0; :2251 on v8.3).
      • _getPing uses _pings[host], which default-inserts an uninitialized PingStats. getMillis() maps UninitializedPingTime to 0, so the result is silently 0 instead of an error.

      Sync source selection is not affected: it already looks pings up via getHostAndPortPriority(). Only the replSetGetStatus output is wrong. The lookup also leaves an unused _pings entry for the main host:port.

      Affected versions

      master, v9.0, v8.3 (checked by reading the code on each branch; reproduced on the 8.3.2 release binary). All platforms.

      Repro (8.3.2, macOS arm64)

      Two-node replica set; both members have priorityPort. A failCommand failpoint on the secondary adds 50 ms to every replSetHeartbeat, so the true ping is about 50 ms rather than localhost's ~0 ms.

      // mongod --replSet pp --port 2801N --priorityPort 2805N --setParameter enableTestCommands=1 (N = 1, 2)
      rs.initiate({_id: "pp", members: [
        {_id: 0, host: "127.0.0.1:28011", priorityPort: 28051, priority: 2},
        {_id: 1, host: "127.0.0.1:28012", priorityPort: 28052, priority: 1}]});
      // on 127.0.0.1:28012
      db.adminCommand({configureFailPoint: "failCommand", mode: "alwaysOn",
        data: {failCommands: ["replSetHeartbeat"], blockConnection: true, blockTimeMS: 50, failInternalCommands: true}});
      // wait ~15s, then on the primary:
      rs.status().members[1].pingMs
      // then setParameter disableReplicationUsageOfPriorityPort: true on both nodes, wait ~15s, check again
      

      Observed:

      WITH priority port:                              pingMs=0  health=1 state=SECONDARY
      WITH disableReplicationUsageOfPriorityPort=true: pingMs=49 health=1
      

      Expected

      pingMs reflects the heartbeat RTT to the endpoint that is actually used for heartbeats (about 49 ms in the repro above).

      Suggested fix

      In prepareStatusResponse, look the ping up with itConfig.getHostAndPortPriority() (matching the heartbeat target and sync source selection) instead of itConfig.getHostAndPort(). A jstest covering pingMs with priority ports would prevent a regression; none of the existing priority port jstests check it.

      Impact

      Diagnostics only: anyone reading pingMs from replSetGetStatus (monitoring, Atlas metrics, support) sees 0 for every member once priority ports are enabled for replication. This is rolling out in Atlas on 8.3+.

            Assignee:
            Unassigned
            Reporter:
            Peter Stefek
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: