-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: DHandles
-
None
-
Storage Engines, Storage Engines - Foundations
-
733.183
-
Storage Execution 2026-08-03
-
None
Per investigations in WT-17617, a large part of the DSC-vs-ASC regression came from repeatedly re-opening stable-table dhandles on the follower every time the system picks up a new checkpoint. Given blind writes, we may not always need the stable handle at all, so this ticket lazily opens the stable table only when actually required, rather than eagerly on every checkpoint pickup.
Correctness bug this also fixes: WT-17254's blind-delete path decides up front, before the operation runs, whether to open the stable table dhandle – and skips it when it expects to take the blind-delete branch, trusting the caller's overwrite=true guarantee that the key is live somewhere. But the operation can still end up taking the normal (non-blind) delete path instead, and that path does need stable to resolve the key. Because stable was never opened – the earlier decision assumed blind-delete and skipped it – the normal delete has no way to find a key that exists only in stable (not in ingest): it surfaces as "not found" and no tombstone is written anywhere. Mongo tolerates the "not found" result and continues. A later reinsertion of the same key then collides with the never-tombstoned original in stable, producing WT_DUPLICATE_KEY / NoSuchKey fasserts on followers/standbys during oplog application.
This ticket's change defers the decision of whether to open the stable table to the point of the actual delete operation, instead of deciding eagerly beforehand. As a result, the normal (non-blind) delete correctly opens stable when needed and writes the tombstone, closing the gap left by WT-17254. The fix is a side effect of the lazy-open redesign, not a targeted bug fix – but it does address the root cause (missing stable lookup on blind-delete miss), not just a symptom.
Observed impact: HELP-97958, SERVER-132631, BF-44855 – clusters running RC1017 (which has WT-17254 but not this fix) hit duplicate-key/fassert errors after index key removal + reinsertion.
Regression coverage: added later in WT-18215 (PR #14325 – https://github.com/wiredtiger/wiredtiger/pull/14325) – this test fails when run against the pre-WT-18159 (RC1017) binary, confirming the fix is what closes the gap.
- causes
-
WT-17254 Require a caller-guaranteed live key for overwrite=true remove on a layered follower cursor
-
- Closed
-
- is related to
-
WT-17254 Require a caller-guaranteed live key for overwrite=true remove on a layered follower cursor
-
- Closed
-
-
WT-17617 Investigate >60% OperationThroughput regression in DSC vs ASC for data_handle_locust workload
-
- Closed
-
- related to
-
WT-18173 Stop modify from falsely returning WT_NOTFOUND when the key exists
-
- Closed
-
-
WT-18215 Blind remove didn't applied when a follower restart without a ingest table
-
- Closed
-
-
WT-18390 Disable blind delete path in __clayered_remove_from_ingest
-
- Closed
-
-
WT-18170 Dhandle scaling Perf: cooperative on demand background open of stable tables
-
- In Progress
-
-
WT-18060 Dhandle scaling Perf: On standby, do selective pickup of individual btree checkpoints
-
- Closed
-
-
WT-18160 Investigate: Perform a second perf analysis of dhandle locust workloads
-
- Closed
-