[SERVER-48024] Have validate compare collection object with on-disk metadata object Created: 07/May/20  Updated: 27/Oct/23  Resolved: 25/Jun/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-48010 Substitute ghost timestamp with no-op... Closed
Assigned Teams:
Storage Execution
Participants:
Linked BF Score: 10

 Description   

Catalog-level data corruption is only detected across restart (or rollback to stable). While a collection validation does read collection and index data from disk, and compares some aggregates to in-memory collection objects (e.g: multikey), the collection object is never checked.

We should add some basic checks to collection validation. Most importantly, verifying the on-disk document exists. Other possibilities (not all may be possible, particularly for background validation):

  • Comparing in-memory multikey values to disk ones
  • collection + index ident mappings
  • UUIDs
  • Namespace value

This check will help catch some errors that are currently only observable on tests that intentionally crash/restart, but may be occurring much more frequently, leading to easier reproducers.



 Comments   
Comment by Daniel Gottlieb (Inactive) [ 25/Jun/20 ]

I'll be darned

Comment by Louis Williams [ 25/Jun/20 ]

daniel.gottlieb, I'm not sure I completely understand the goal of this ticket. We already have checks in validate() that would fail if the entry in the durable catalog is missing. For example. Do you mind clarifying what you wanted out of this?

Generated at Thu Feb 08 05:15:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.