[SERVER-38588] Hybrid index builds do not work when applied concurrently with prepared transactions on secondaries Created: 12/Dec/18 Updated: 29/Oct/23 Resolved: 08/Mar/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance, Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.9 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Randolph Tan | Assignee: | Eric Milkie |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | open_todo_in_code | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Storage NYC 2019-03-11 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 22 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Original text:
Because transactions yield their locks on secondaries ( In this example, a background index build on {a: 1} is concurrent with an insert of a document {a: 0} in a transaction while applied on a secondary.
On a primary, our locks prevent this from happening, but because an index build can complete while a prepared transaction is active, we can lose writes into building indexes. edit: louis.williams |
| Comments |
| Comment by Githook User [ 08/Mar/19 ] |
|
Author: {'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'}Message: This will prevent hybrid index builds from corrupting an index on secondary nodes if a prepared transaction becomes prepared during a build but commits after the index build commits. |
| Comment by Eric Milkie [ 28/Feb/19 ] |
|
Note that this work will in effect stall replication when a prepared transaction is encountered that conflicts with a background index build, until such build completes. |
| Comment by Eric Milkie [ 28/Feb/19 ] |
|
I removed this ticket from the Simultaneous Index Builds epic, as we came up with a different solution that will remove the dependency on that project's completion. The work for this ticket will be the following: |
| Comment by Eric Milkie [ 04/Jan/19 ] |
|
Moved this ticket to the Simultaneous Index Builds epic, as that project will be the work to fix this problem completely. The hybrid project has provided the interface for Simul. to call at the correct times. |
| Comment by Randolph Tan [ 13/Dec/18 ] |
|
Note: blacklisted background_index_multikey.js here and will also add it to new sharding suites as well. Will tag the comment with this ticket number when adding the new blacklists. |
| Comment by Randolph Tan [ 12/Dec/18 ] |
|
louis.williams took a first look and suspects that there might be a race in replicating the background index build and replicating the documents. |