-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Replication, Replication - Oplog and Recovery
-
0
-
None
-
None
-
None
-
None
-
None
-
None
-
None
SERVER-137256 wrapped the asCluster()'s (jstests/libs/replsettest.js) connectionStatus call in retryOnNetworkError(..., ReplSetTest.kDefaultRetries, 1000), so suites that kill or step down nodes survive a dropped connection to a node that comes back.
BF-47145 showed a side effect. ReplSetTest.waitForState's inner _waitForIndicator call is meant to recover from errors. It probes all nodes with _callHello and picks a live one. It then runs a retry loop against the same connection and never re-probes. If that connection is to a node that is dying and won't return, all 240 retries run (about 13 minutes in the BF logs), exceeding waitForState's 600s timeout.
In stepdown_fassert_on_explicit_overridden_rstl_timeout.js, waitForState(DOWN) was being called on a node that had just stepped down. If the node's shut down process and waitForState call overlapped, this node could be picked by _waitForIndicator. _waitForIndicator would then stall on the permanently down node. BF-47145 was fixed in stepdown_fassert_on_explicit_overridden_rstl_timeout.js by waiting for the primary to stop accepting connections before waitForState(DOWN).
Acceptance Criteria:
Decide whether asCluster()'s retry should be changed so that other tests which poll a node while it crashes or shuts down cannot hit the same stall.
- is related to
-
SERVER-137256 retry connections when verifying repl dbhash
-
- Closed
-