-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Execution
-
Fully Compatible
-
ALL
-
Storage Execution 2026-11-09
-
0
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The second phase of jstests/noPassthrough/index_builds/index_build_killop_secondary_before_commit.js (killOp on the secondary after the index build voted for commit, failpoint hangIndexBuildAfterSignalPrimaryForCommitReadiness) intentionally crashes the secondary with fassert 51101. However the test issues the killOp with assert.commandWorked(secondaryDB.killOp(opId)) before checking for the crash. The killOp interrupt is what causes the crash, so there is an inherent race between the conn thread flushing the killOp response and the index build thread aborting the process. If the abort wins, the shell sees a network error ("Connection closed by peer") and the test fails even though the server behaved exactly as designed.
The race exists on every platform but is effectively only losable on slow or CPU-starved hosts, which is why it surfaced on windows-debug-suggested (the system log showed "CPU constraint detected (sustained >90% for 5+ minutes)").
Evidence (BF-47023, commit 7ad00b7b, windows-debug-suggested)
- IndexBuildsCoordinatorMongod-0 hit fassert 51101 at index_builds_coordinator.cpp:3625 with "Interrupted ... caused by :: operation was interrupted" at 2026-10-30T07:36:54.446.
- conn1 logged "Killed operation" for opId 46081 in the same millisecond, so killOp did execute server-side, but the process aborted before the response reached the shell.
- The shell then failed at index_build_killop_secondary_before_commit.js:67 (assert.commandWorked(secondaryDB.killOp(opId))) called from line 155 (the
shouldSucceed=false phase).
Root Cause
After the build votes for commit, ReplIndexBuildState::canVoteForAbort() returns false. When the killOp interrupt unwinds the build, IndexBuildsCoordinator::_cleanUpTwoPhaseAfterNonShutdownFailure cannot vote for abort, and since the node is a secondary it fasserts 51101. This is intended behavior and is what the test wants to verify. The bug is only in the test expecting a command response from a server it is deliberately crashing.
Additional Notes
- jstests/noPassthrough/index_builds/index_build_killop_secondary_after_commit.js uses the identical assert.commandWorked(killOp) then expect-crash pattern and has the same latent race. It is currently excluded on Windows via incompatible_with_windows_tls with a TODO pointing at
SERVER-116978. Consider fixing it the same way and removing its Windows ban. - Once this fix lands, the Windows ban added under SERVER-XXXXX (disable ticket) for index_build_killop_secondary_before_commit.js should be reverted.
- is related to
-
SERVER-138285 Disable index_build_killop_secondary_before_commit.js on Windows
-
- Closed
-
- related to
-
SERVER-116978 Tests that expect server crashes will spuriously time out on Windows
-
- Closed
-