Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-66171

Make checkpoint operations persist derived metadata values that have change since the last checkpoint

    • Type: Icon: Task Task
    • Resolution: Won't Do
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage Execution

      This will take some timestamp coordination.

      The derived metadata updates cannot have valid-at timestamps earlier than the checkpoint's recovery timestamp, aka the stable_timestamp at which it checkpoints. Additionally, the derived metadata valid-at timestamps must have reached the majority committed point, so that replication oplog recovery is guaranteed to replay up to those timestamps (as opposed to stopping short, leaving us with an incorrect count/dataSize that we cannot "undo").

      Derived metadata values that have not changed since the last checkpoint need not be written out in the checkpoint. We should probably build some kind of registry for namespaces that need to be persisted?

      We'll need to either stall the stable_timestamp from moving forward (so the checkpoint timestamp doesn't move forward) or specify a stable_timestamp at which the checkpoint should be taken. I favor the latter, but have not investigated whether there would be any WT ramifications with the change, or that the MDB won't weirdly not work with it for some subtle reason.

      Clean shutdown should save the latest derived metadata values before the checkpoint. In a silent system, the checkpoint should be entirely clean, all oplog applied, count/dataSize up to date.

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            dianna.hohensee@mongodb.com Dianna Hohensee (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: