Change streams: renameCollection with dropTarget:true must emit a drop event for the overwritten target (SERVER-53478)

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      When renameCollection is called with dropTarget: true and the target collection already exists, the change-stream pipeline emits only a rename event. The implicit drop of the overwritten target is invisible to:

      • Watchers of the target collection (no drop, no invalidate; cursor left dangling).
      • Database- and cluster-scoped watchers (no drop alongside the rename).

      Downstream consumers that rely on drop events to invalidate materialised views, projection caches, or search indexes silently skip the overwritten collection.

      Reproduction

      See attached regression test jstests/change_streams/rename_drop_target_emits_drop.js.

      const src = db.getSiblingDB('t').src;
      const dst = db.getSiblingDB('t').dst;
      src.insertOne({_id: 1});
      dst.insertOne({_id: 1});
      
      const watcher = dst.watch();
      src.renameCollection('dst', true /* dropTarget */);
      
      // Expected: watcher.next() returns {operationType: 'drop', ns: {db:'t', coll:'dst'}},
      // then {operationType: 'invalidate'}.
      // Actual:   watcher.next() blocks forever; no events are produced.
      

      Desired behaviour

      1. A change stream open on the about-to-be-overwritten target receives a synthetic drop event followed by invalidate.
      2. A db- or cluster-scoped change stream observes drop (overwritten target) ordered strictly before rename (src → dst), both carrying the same clusterTime / wallTime.
      3. A plain dropTarget: true rename whose target does NOT exist emits only rename (no spurious drop). The synthetic event must be gated on o.dropTarget actually being present in the oplog entry.

      Proposed fix

      In src/mongo/db/pipeline/change_stream_event_transform.cpp, the kCommand → o.renameCollection branch (lines 474-495) emits one document per oplog entry today. Extend it to buffer a synthetic drop document when o.dropTarget is present, ordered before the rename. Resume-token ordering preserved via a new kSyntheticDropFromRename eventIdentifier discriminator on the resume token.

      Approximate change size: ~30 lines in change_stream_event_transform.cpp, 1 line in resume_token.cpp, plus the new regression jstest and an extended unit-test fixture in document_source_change_stream_test.cpp:1880.

      Tests

      • jstests/change_streams/rename_drop_target_emits_drop.js (new, 165 lines) — three cases: collection-scoped, db-scoped (with $match on drop|rename), negative no-target.
      • document_source_change_stream_test.cpp::TransformRenameShowExpandedEvents (existing, extend) — assert synthetic drop precedes rename in transformer output.

      Related

      • SERVER-53478 — change stream does not see drop of collection in renameCollection
      • Companion design doc in this PR: src/mongo/db/pipeline/RENAME_DROP_EVENT_FIX.md

            Assignee:
            Unassigned
            Reporter:
            Mehar Grewal
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: