-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Checkpoints, DHandles
-
None
-
Storage Engines
-
111.977
-
None
-
None
Follower checkpoint pick-up spends most of its time in _disagg_apply_checkpoint_meta updating local file: metadata for every layered table. For each table we only change the checkpoint= field, but we currently rebuild the entire metadata string with _wt_config_collapse, which re-walks every config key and re-searches the config array. That cost scales linearly with table count and dominates pick-up latency.
Why this matters
In disaggregated deployments, secondaries (followers) adopt the primary’s checkpoint via reconfigure(checkpoint_meta=…). Until that pick-up finishes, the follower is not caught up on the latest durable shared state.
At large catalog sizes (thousands of collections / layered tables), pick-up already takes on the order of ~1.4s for 5K tables on the update path alone. That latency sits on the critical path for:
- Failover / step-up readiness
- Replica catch-up after checkpoint advance
This is pure CPU/metadata work, not IO to object storage. Cutting it improves availability characteristics of large disagg clusters without changing durability semantics.
Proposed fix
When both local and shared metadata already have the file: entry, replace only the checkpoint= value with a single-pass rewrite instead of __wt_config_collapse. Semantics stay the same (only the checkpoint cookie moves; see existing FIXME-WT-14730).
Evidence (PoC)
Workload: mutate all N layered tables on the leader, then time follower update pick-up (disagg_apply_checkpoint_meta_time).
5K tables:
- Before~1399 ms
- After (single-pass rewrite) ~217 ms
- Speedup 84%
Experiment artifacts: exp_apply_meta_opt/ (fix01_checkpoint_replace.patch).
Acceptance criteria
- Update pick-up with N dirty existing tables no longer calls __wt_config_collapse on that path disagg_pick_up_file_meta_updated still tracks ~N updates
- Existing layered checkpoint pick-up tests pass (test_layered_checkpoint*, especially re-pick-up / update paths
- is related to
-
WT-14730 Validate immutable metadata config fields on checkpoint pickup
-
- Open
-
- related to
-
SERVER-132164 AdaptiveCheckpointPolicy (SERVER-120876) causes multi-second read blackouts on disagg standbys
-
- Open
-