Problem:
If a reader is stalled on a prepared update, a few things can change underneath it resulting in the clearing of iteration flags.
- The key the stable cursor is parked on can be deleted when a new checkpoint is picked up
- The reader's snapshot can be refreshed
- The read timestamp can change
In all three cases we currently hit an assertion where we expect the iteration flags to still be set whenever the ingest cursor is keyless. That is true when the stall begins, but the three changes above clear the iteration flags while the stall is still in progress.
In this particular BF, the problem is the first one (checkpoint). There is no resetting of the snapshot, and the read timestamp is fixed.
Context:
This problem was highlighted in BF-43990. One sequence of steps to hit the assertion is:
Keys:
ingest : 1, (3 prepared), tombstone(4), 5
stable : 4, 6 (older checkpoint)
Sequence:
Walk (read at a fixed timestamp):
1. next() -> returns 1
ingest @1, stable @4
2. next() -> ingest 1->3, but 3 is PREPARED -> stop and wait
ingest keyless @3, stable parked @4
(ingest's tombstone(4) is further along -- not reached yet)
3. (background) a new checkpoint with 4 deleted is produced -> follower picks it up
ingest's tombstone(4) still exists.
4. next() -> reopen stable onto the new checkpoint
dup_position(4) -> NOTFOUND (4 is gone in the new checkpoint)
-> stable UNPOSITIONED, iterate flags cleared
5. (prepare 3 resolves) next() -> ingest 3 -> tombstone(4) [skipped] -> 5
stable is still unpositioned -> 6 is never returned
Returned : 1, 5
Should be: 1, 5, 6
Timestamps:
ts 10: key 3 prepared
ts 20: leader deletes key 4
ts 30: read_timestamp = 30
Definition of Done:
The scope of this ticket is to address BF-43990 directly.
- Prevent from stable advancing if there is a prepare conflict
- Have an assertion on the snapshot/timestamp path
- Write python tests
- is related to
-
WT-17969 Follower layered cursor aborts in __clayered_reopen_stable when a read-timestamped reader's parked stable key vanishes from a legitimately-adopted newer checkpoint
-
- In Code Review
-
-
-
- Closed
-
- related to
-
-
- In Code Review
-
-
-
- Closed
-