ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1024
    • Affects Version/s: None
    • Component/s: None
    • None
    • Cluster Scalability
    • Fully Compatible
    • ClusterScalability 14Sep-28Sep, ClusterScalability 28Sep-12Oct
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Impact

      Resharding C++ tests can hang in shutdownExecutorPool() or hit teardown-time invariants when the test exits with a donor, recipient, or coordinator PrimaryOnlyService instance that has not fully completed.

      Related failures include BF-43436, BF-43491, BF-43612, and BF-44484.

      Root Cause

      ReshardingCoordinatorServiceTestCommon::tearDown() never steps down the coordinator instances, and it calls _registry->onShutdown() only after ConfigServerTestFixture::tearDown(). That base teardown shuts down the executor pool, ShardRegistry and CatalogCache, and clears ShardingCatalogManager and Grid, so coordinators can keep running against torn-down sharding state. ReshardingDonorRecipientCommonTest already uses the correct order: step down, shut down executors, join instances, then tear down the fixture. Its per-test interrupt() calls are redundant.

      Fix

      Make each fixture responsible for fully quiescing the state machines it creates.

      • In the coordinator fixture: step down, shut down the executor pool, then call onShutdown() before ConfigServerTestFixture::tearDown(). Remove the redundant per-test interrupt() calls.
      • Do not require individual test bodies to interrupt or run state machines to completion solely for safe fixture teardown.

      Testing

      • Run ProcessDonorFields* tests without per-test cleanup and verify teardown completes.
      • Run CreateRecipientServiceInstance without its per-test interrupt() and verify teardown neither hangs nor triggers the recipient state-transition invariant.
      • Add a coordinator test that returns while a mock fetch is blocked, without explicitly calling stepDown() or completing resharding.
      • Repeat the affected tests under TSAN to exercise teardown races.

      Related

            Assignee:
            Abdul Qadeer
            Reporter:
            Abdul Qadeer
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: