[SERVER-40854] Hybrid index builds should reposition cursors to the beginning of the side table after yielding Created: 26/Apr/19 Updated: 29/Oct/23 Resolved: 03/May/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.11 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Louis Williams |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Sprint: | Storage NYC 2019-05-06 | ||||||||||||
| Participants: | |||||||||||||
| Linked BF Score: | 64 | ||||||||||||
| Description |
|
During the first drain phase of hybrid index builds, operations are applied in batches from the side table record store into the index. At the end of each batch (1000 operations by default), the side table cursor is saved and restored. This restore may reposition on new records, but not necessarily the newest records. This can cause batch applications after yield to apply causal operations out of order. For example, imagine two writer threads, A and B, writing into the side table and an index build thread. In this example, thread A inserts then deletes the key {a: 1}. Thread B inserts an unrelated key.
In this scenario, it is possible to insert the key {a: 1} into the index even after it has been deleted. The fix for this bug should be to reposition cursors at the beginning of the record store after every yield to guarantee all causally related records are applied in order. |
| Comments |
| Comment by Githook User [ 03/May/19 ] |
|
Author: {'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}Message: Re-enables failing StorageInterfaceImplTest test cases. |
| Comment by Benety Goh [ 03/May/19 ] |
|
Re-opening to fix index build interceptor |
| Comment by Githook User [ 03/May/19 ] |
|
Author: {'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}Message: |
| Comment by Githook User [ 02/May/19 ] |
|
Author: {'name': 'Louis Williams', 'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com'}Message: This also adds tunable parameters 'maxIndexBuildDrainBatchSize' and |