[SERVER-41977] Index lookups may conflict with prepared transactions Created: 27/Jun/19 Updated: 18/Oct/23 Resolved: 12/Aug/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Louis Williams |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | isfz | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Sprint: | Execution Team 2021-02-08, Execution Team 2021-02-22, Execution Team 2021-03-08, Execution Team 2021-08-23 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 8 | ||||||||||||||||
| Description |
|
The way we query indexes internally is incompatible with prepared transactions. According to WT_CURSOR::search_near, it will return "the record matching the key if it exists, or an adjacent record." On an index, given the keys "A1" and "B2", and a query for "B", it is completely within definition for search_near to return "A1" instead of "B2". In this example, the indexed value is "B" and the RecordId is 2, which we concatenate to form the key we insert into the WiredTiger table, "B2". We then are only able to query for "B", which is not the complete key. That means if there is a prepared update on "A1", an update or delete on "B" (which first requires a lookup) may hit a prepare conflict, even though "B2" exists and is a closer match.
This is a concern on secondaries and initial-syncing nodes, because they may encounter prepare conflicts that did not occur on the primary. We should consider partially-reverting |
| Comments |
| Comment by Louis Williams [ 12/Aug/21 ] |
|
We have decided that we will not fix this problem. Prepare conflicts are meant to resolve as quickly as possible, so any visible stalls caused by this behavior should not last for long periods of time. |
| Comment by Louis Williams [ 12/Apr/21 ] |
|
In |
| Comment by Alexander Gorrod [ 01/Jul/19 ] |
|
Thanks for the update louis.williams |
| Comment by Louis Williams [ 28/Jun/19 ] |
|
alexander.gorrod Yes, I think that will still be a problem with non-unique indexes, so I don't think an API change is required at the moment. For the time being, we'll keep this ticket open to track the problem, but the present work will be
|
| Comment by Alexander Gorrod [ 28/Jun/19 ] |
|
I thought there was an issue with search_prefix in a non-unique index i.e: There could be two "B" entries, and the one you aren't looking for could be prepared. I'll stand by for an answer, but there is significant work involved in WiredTiger in either this solution or the other one we talked about which was allowing for a key to be retrieved after a prepare conflict, and allowing for controlling cursor traversal behavior around prepare conflicts. |
| Comment by Louis Williams [ 27/Jun/19 ] |
|
alexander.gorrod, milkie, geert.bosch should we reconsider the idea of having a "search_prefix" functionality? |