Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-98175

Add a change streams consistency checker in tests

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      Add a stateful verifier that checks to make sure that change streams events are consistent with each other. One possible way to use this verifier is to open a change streams cursors at the beginning of tests/suites (has to be done at the beginning since change streams doesn't yet support starting from the beginning of oplog on all shards as of v8.0 and it will be trickier to pick a timestamp to start with) and perform the checks while iterating over it.

      Motivation: Resharding changes the shard key pattern and also performs ddl changes on multiple shards. We had seen some inconsistencies that arose in the past like SERVER-58857, SERVER-66006 and SERVER-97044.

      Some ideas of things to check:

      • Keep track of uuids from create/rename collection events. Throw an error if we see an op that refers to a uuid we have never seen before or should have been dropped (this is most likely a sign of change events being out of order - like insert showing up before the create).
      • Keep track of namespaces and throw an error if an op shows up with a new namespace before create or rename or after it was dropped.
      • Keep track of the shard key pattern from the documentKey field. Throw an error if we see the pattern changed without or before a shardCollection/refineShardKey/reshardCollection event.

      Another possible thing to incorporate on the checker is to randomly decide to open a new cursor and use the resume token to continue instead of using the existing change stream cursor.

      Checking full document consistency will be trickier and complicated. Can we rely on the existing mongosync test to do this verification?

            Assignee:
            Unassigned Unassigned
            Reporter:
            randolph@mongodb.com Randolph Tan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: