-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Layered Tables
-
Storage Engines - Foundations
-
5.632
-
None
-
None
The WT_LAYERED_TABLE_STEP_DOWN_CREATED flag lives in the plain (non-atomic) uint8_t flags word of WT_LAYERED_TABLE and is accessed concurrently without a common lock:
- Cleared in __disagg_mark_btrees_readonly_then_step_down (src/conn/conn_layered.c) during reconfigure to follower, holding the schema, checkpoint and handle-list-read locks.
- Read via plain F_ISSET from cursor operations in _clayered_enter and _clayered_update_stable (src/cursor/cur_layered.c), which hold none of those locks.
This is a data race by the C11 memory model (undefined behavior) and will be reported by TSan.
Why it is not currently dangerous:
- The flags word is a single byte, so the load cannot tear.
- The clear happens-before the leader = false release store at the end of the step-down walk, and cursors resolve their role with an acquire load, so any cursor that observes the follower role also observes the cleared flag. A cursor observing the stale leader role gets the leader-era flag value, which is consistent.
- The one problematic interleaving — a cursor sees the stale leader role but the early-visible cleared flag, and attempts to open a stable constituent that does not exist — is absorbed by __clayered_ignore_missing_stable: the failed open acquired the schema lock, so by that point the step-down timestamp is set or the role has flipped, the ENOENT/WT_NOTFOUND is swallowed, and the cursor correctly proceeds ingest-only.
- There is no concurrent read-modify-write on the shared flags word: sweep never closes layered dhandles, and all other close paths (__wt_schema_close_layered clearing WT_LAYERED_TABLE_OPEN) run under the schema lock, which the step-down walk also holds.
Why it should still be fixed:
The benignness is situational, resting on an unrelated release/acquire pair, an error-swallowing helper on the failure path, and the sweep exemption for layered dhandles. None of these were designed to protect this flag, and any of them can change independently (e.g. FIXME-WT-16982 proposes letting sweep close layered dhandles, which would introduce a lost-update race on the flags word). A racy plain load is also undefined behavior, so the compiler is not required to preserve the naive codegen the analysis above assumes. The flag accesses should be converted to atomic flag operations, following the approach of WT-18015 (which did the same for concurrently modified WT_BTREE flags).