-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Metadata
-
None
-
Storage Engines - Persistence
-
360.599
-
StorEng - Defined Pipeline
-
None
Provide an efficient mechanism for picking up checkpoints by finding a difference between the follower's current shared metadata table and the shared metadata table from the new checkpoint. As explained in WT-18819, we can do generate the diff very efficiently by leveraging the copy-on-write nature of WiredTiger btrees.
This will allow us to identify the following cases without having to scan through the entire shared metadata table:
- Newly created tables
- Dropped tables
- Tables with different file: metadata, e.g., due to checkpoints or drop/recreate.
We will still need to fall back to the current mechanism that reconciles the differences between the local and the shared metadata in the following cases:
- Startup
- First checkpoint pickup after step down
- When recovering from a failed checkpoint pickup
We will, however, be able to use this mechanism in the steady state for followers.