Fix flake in PrimaryDrivenResumableIndexBuildTest._waitForBuildOutcome caused by RamLog rollover

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • Fully Compatible
    • ALL
    • 0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      PrimaryDrivenResumableIndexBuildTest._waitForBuildOutcome in primary_driven.js is flaky and intermittently times out after 300s even when the resumed index build actually completed successfully on the new primary.

      The helper polled the new primary's logs via checkLog.getGlobalLog(newPrimary) looking for either log id 20663 (build completed) or 11130400 (build aborted after resume failure). checkLog.getGlobalLog reads the in-memory global RamLog ring buffer, which is capped at 1024 entries (_globalMaxLines in ramlog.cpp). Under verbose step-up / recovery traffic, more than 1024 log lines can be emitted between the build's completion and the test's first poll — at which point the 20663 line has been overwritten and the helper waits 300s for a signal that will never reappear.

      Observed in BF-43616 on resumable_multi_phase_end_unclean_restart.js:

      d22491 emitted log id 20663 for the resumed build at 09:06:39.100 (log line 8821).
      Test framework's first _waitForBuildOutcome poll was at 09:07:00 (log line 15090) — ~21s later.
      In that gap d22491 emitted 1628 log lines, overwriting the 20663 entry.
      The helper then polled every 1s for 300s and timed out.
      The failure is not a server bug or a replication race; it is a test-helper bug — relying on a bounded ring buffer for a signal that may be evicted before the first read. File-based logs are not a workable fallback because mongod in tests only writes to disk when jsTestOptions().alwaysUseLogFiles is set (single hit at jstests/libs/replsettest.js:3395).

      This ticket addresses the flakyness by replacing the log-scraping logic in _waitForBuildOutcome with state-based polling that does not depend on the RamLog contents.

            Assignee:
            Matt Olma
            Reporter:
            Matt Olma
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: