Implement KEK read path mechanism

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines - Foundations
    • SE Foundations - 2025-12-19
    • 5

      Context
      After WT-16052 and WT-16053, we should have an extension example module and the ability to read/write to the turtle file. As such, we now implement the read KEK mechanism. During checkpoint, if we encounter a new key blob we need to write the new lsn reference to the turtle file.

      When a checkpoint occurs (triggered by a WT_SESSION->checkpoint call by mongod, the following steps occur:

      • WT writes dirty pages for btrees (these are PALI writes) these return LSNs - the LSN for the root page of each tree is started (as part of a checkpoint cookie) in the metadata btree.
      • WT writes the metadata btree, the last Btree to be written (via PALI writes) - the LSN for the root page of the metadata btree is saved (used in a later step)
      • WT calls the new API get_key_blob to retrieve the current KEK blob. If the call indicates that no changes were made (since the previous get_key_blob) call, then steps 4 and 5 are skipped, and we use the previously saved LSN for step 6.
      • WT puts a small header in front of the blob and writes the result to the KEK page (via PALI write) the LSN is used in step 6.
      • WT calls the get_complete API to return the LSN to the encryption layer, and to discard the memory used to store the blob.
      • WT creates the turtle page. Within the turtle information is the saved LSN for the metadata btree, also the LSN for the KEK page.
      • WT writes the turtle page via PALI write. The LSN returned is part of a string given to MongoDB, as it uniquely identifies the checkpoint.

      Leader/Follower mode

      • Startup (on existing WT): Verify that KEK blob is re-used
      • Checkpoints using same KEK blob: KEK blob should be reused for each checkpoint

      Follower Mode

      • Verify checkpoint picks up correct KEK blob from metadata

      Defintion of Done

      • WiredTiger utilises get_key_blob and get_key_complete within the checkpoint. If the has_changed variable has changed, it writes the KEK blob into the turtle file.
      • Persists the KEK blob calling into PALI
      • Tests scenarios are created using either example, or mock module

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: