Include event provenance in change streams

    • Type: New Feature
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      For mongosync and its embedded verifier, it would be extremely helpful if we could identify the provenance of change events. In particular, it would be very useful if we could identify writes from the TTL monitor.

      During mongosync's critical section, we require that all user writes are stopped to the source and destination cluster. The embedded verifier watches the change stream during this period to identify incorrect writes, but there are still two sources of writes that we expect to happen: the TTL monitor deleting expired documents, and writes from mongosync itself.

      The verifier has some heuristics to identify these writes (checking if it's a delete from a collection with a TTL index), but they are imperfect, which makes it difficult to be fully confident in the correctness of the verifier. An ideal solution would be some field in the change stream that identifies the source of the writes, so that we could confidently identify the source of the write.

      (I realize that this will likely involve work from multiple teams.)

              Assignee:
              Unassigned
              Reporter:
              Michael McClimon
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: