[SERVER-78365] Investigate removing ignore_prepare=force Created: 22/Jun/23  Updated: 06/Feb/24

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Geert Bosch Assignee: Gregory Noma
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-64515 Remove claims of prepare behavior fro... Open
is related to SERVER-68380 Investigate MongoDB code that can use... Closed
Assigned Teams:
Storage Execution
Sprint: Execution NAMR Team 2023-07-24, Execution NAMR Team 2023-08-07, Execution Team 2023-11-13, Execution Team 2023-11-27, Execution Team 2024-03-18
Participants:

 Description   

This option significantly complicates reasoning about behaviors around prepared transactions, so removing it should be high on our list of behaviors to remove.



 Comments   
Comment by Louis Williams [ 20/Nov/23 ]

I don't see a path forward to eliminate this API as long as we want a safety mechanism from WiredTiger to prevent us from introducing data corruption bugs.

Comment by Louis Williams [ 01/Nov/23 ]

There's quite a bit to discuss here. But first I'll just provide background.

Data corruption bugs like SERVER-39074 made us realize that it was very risky to perform writes while ignoring prepared updates, and almost never safe. The ability to ignore prepared updates was just an optimization for read-only operations. This prompted us to ban this behavior by default (WT-4580) and then use the ignore_prepare=force override for cases where we made sure it was safe to read and write while ignoring prepare conflicts.

As of today, we have a few users of this override:

This override is required unconditionally on secondaries due to the problems described in SERVER-40176 and SERVER-40167. That is, we can hit prepare conflicts on secondaries that would not have happened on the primary, just by bad luck. We can accidentally find a key in a WT table that is next to a prepared update.

If we don't like the current API, it seems like we need to rethink the API a bit to deliver something that works better for our current code. We still want to respect these requirements:

  • We want to be able to ignore prepared updates for most read operations while also having safeguards to avoid future data corruption bugs.
  • Unless we only have queries spill in a new RecoveryUnit, we'll need to be able to read and write within a WT transaction that is ignoring prepare conflicts. This may actually become a reality with SERVER-81331.
  • Unless we want to re-open SERVER-40176 and SERVER-40167 to address prepare conflicts on unrelated keys, we will also need to continue ignoring prepare conflicts and allowing writes on secondaries
Generated at Thu Feb 08 06:38:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.