v2 change stream stalls indefinitely after startAfter(rename-invalidate) + movePrimary

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Won't Do
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • QE 2026-07-06, QE 2026-07-20
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      A collection-level v2 change stream (featureFlagChangeStreamPreciseShardTargeting) can hang indefinitely after the following sequence:

      1. Open a change stream on db.coll.
      2. Rename coll → coll2 (produces rename + invalidate events on the stream).
      3. Move the database primary to a different shard (movePrimary).
      4. Reopen the stream with startAfter(invalidateToken).
      5. Recreate coll on the new primary and insert a document.

      The stream never observes the insert. It silently blocks until the connector times out.

      Root Cause

      When the v2 shard targeter receives a MovePrimaryControlEvent (written by the old primary as a noop oplog entry), it calls handlePlacementRefresh() to re-evaluate which shards to watch.

      At the time of movePrimary, the collection placement for coll is empty — the collection was renamed away at an earlier timestamp. The existing code treated empty collection placement as equivalent to database absent, and would:

      1. Open a cursor on the config server.
      2. Switch the event handler to DbAbsentState.

      DbAbsentState only wakes on a DatabaseCreatedControlEvent (an insert into config.databases). Because the database already exists, this event never fires — the stream stalls permanently.

      The bug is in ChangeStreamShardTargeterDbPresentStateEventHandler::handlePlacementRefresh() in src/mongo/s/change_streams/change_stream_db_present_state_event_handler.cpp.

      Fix

      Before falling through to the DbAbsent transition, check database-level placement when collection placement is empty (for collection-level streams only). If the database is present (non-empty dbShards), redirect the cursor to the database primary shard and remain in DbPresent state. This allows the stream to observe the collection once it is recreated on the new primary.

      Tests

      • Unit tests: 5 new test cases + 2 updated existing tests in collection_change_stream_db_present_state_event_handler_test.cpp covering both NamespacePlacementChangedControlEvent and MovePrimaryControlEvent paths (strict mode, ignoreRemovedShards mode, and NotAvailable → kSwitchToV1).
      • Integration reprojstests/change_streams/change_stream_collection_v2_start_after_rename_move_primary.js — a minimal end-to-end test on a 2-shard cluster that reproduces the hang without the fix and passes with it.

            Assignee:
            Daniel Tabacaru
            Reporter:
            Daniel Tabacaru
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: