[SERVER-59842] Setting wildcard index as multikey can cause prepare conflicts Created: 08/Sep/21  Updated: 11/Nov/21  Resolved: 11/Nov/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Samyukta Lanka Assignee: Judah Schvimer
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File repro.js    
Issue Links:
Depends
Related
is related to SERVER-40167 Index key removal should not encounte... Closed
is related to SERVER-41766 Secondary may encounter prepare confl... Closed
Operating System: ALL
Steps To Reproduce:

1. Create a wildcard index
2. Make the index multikey as part of a transaction
3. Prepare the transaction
4. Insert a doc into the same collection that would make the index multikey

Participants:
Linked BF Score: 15

 Description   

We can cause a prepare conflict if we set a wildcard index as multikey as part of a prepared transaction and then try to set it as multikey through a write separate from the transaction.

The conflict comes from when we insert keys into the index itself for wildcard indexes (multiMetadataKeys is populated by the wildcard_key_generator). I think that is done in the same wuow as the transaction, which means that it will generate a prepare conflict if we try to write on top of it.

I suspect this would be resolved if we did these writes in a side transaction block, like when we set the index as multikey.



 Comments   
Comment by Judah Schvimer [ 11/Nov/21 ]

Closing per samy.lanka's suggestion.

Comment by Samyukta Lanka [ 09/Sep/21 ]

I'm starting to question whether this is worth fixing. The prepare conflict should resolve once we abort or commit the prepared transaction. If the prepared transaction exists indefinitely, then I think we'll have other issues to deal with. SERVER-41977 and SERVER-40167 are two other similar situations where we've decided not to fix prepare conflicts in the past, although I think this issue is not as rare as both the linked issues. Marking this ticket as needing triage to discuss further.

Comment by Samyukta Lanka [ 09/Sep/21 ]

It looks like doing the writes in a side transaction block will cause other issues with transactions. Let's say we try to use the multikey path from the wildcard index to find some documents in the same transaction that set the index as multikey. The side transaction block writes at a timestamp not included in the readTimestamp for the transaction, which means that if we need to access the keys written to the index within the same transaction (like when serving a read), we'll hit an issue due to snapshot isolation.

Generated at Thu Feb 08 05:48:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.