Have validate compare collection object with on-disk metadata object

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Works as Designed
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Storage
    • None
    • Storage Execution
    • 10
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Catalog-level data corruption is only detected across restart (or rollback to stable). While a collection validation does read collection and index data from disk, and compares some aggregates to in-memory collection objects (e.g: multikey), the collection object is never checked.

      We should add some basic checks to collection validation. Most importantly, verifying the on-disk document exists. Other possibilities (not all may be possible, particularly for background validation):

      • Comparing in-memory multikey values to disk ones
      • collection + index ident mappings
      • UUIDs
      • Namespace value

      This check will help catch some errors that are currently only observable on tests that intentionally crash/restart, but may be occurring much more frequently, leading to easier reproducers.

              Assignee:
              [DO NOT USE] Backlog - Storage Execution Team
              Reporter:
              Daniel Gottlieb (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: