[SERVER-44507] Hybrid index build is able to commit (acquire stronger mode locks) for a collection that contains prepared documents. (4.2 only) Created: 08/Nov/19 Updated: 29/Oct/23 Resolved: 08/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 4.2.1 |
| Fix Version/s: | 4.2.6 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Suganthi Mani | Assignee: | Eric Milkie |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2019-12-02, Execution Team 2020-02-24, Execution Team 2020-03-09, Execution Team 2020-03-23, Execution Team 2020-04-06, Execution Team 2020-04-20 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Since hybrid index build collection scan phase can yield, it can violate the locking contracts of prepared transactions. Consider a below sequence. 1) Hybrid index build gets started on secondary for a collection A. And, collection phase of hybrid index build on secondary gets yielded. (Makes all the mongoDB and document level locks to get released - abandons the snapshot). Even, if we make the collection scan phase of index build to not yield, we would still can face the same problem mentioned above. Since, after Basically we are trying to do a following sequence that is not valid
P.S: The above sequence is prevented during secondary oplog application by this. Notes:
|
| Comments |
| Comment by Githook User [ 08/Apr/20 ] |
|
Author: {'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'}Message: |
| Comment by Eric Milkie [ 12/Mar/20 ] |
|
New plan is to detect this situation at index completion time (since we will see the number of writes to the side table will not be equal to the number read out of the side table), and we can then block with no locks held until the next prepared transaction commit, and then try draining the side table again. |
| Comment by Eric Milkie [ 05/Mar/20 ] |
|
I have come to the realization that my original solution won't work, as it's too late to block the prepare command at the time of its receipt, since the transaction has already acquired locks, so it will just block the index build forever. This technique works on a secondary because we can block prior to doing any locking for the transaction. |
| Comment by Eric Milkie [ 11/Nov/19 ] |
|
Note that today's solution in 4.2 that blocks prepare on secondaries already has this stress-inducing behavior. I expect it indeed causes a prepared transaction attempt to time out. |
| Comment by Judah Schvimer [ 11/Nov/19 ] |
|
I'm nervous to block prepare for that long. I think realistically it will lead to the transaction aborting due to a timeout, which maybe is fine in rare circumstances. |
| Comment by Eric Milkie [ 08/Nov/19 ] |
|
I think the best way to solve this is to apply a similar solution as the one I used in |
| Comment by Suganthi Mani [ 08/Nov/19 ] |
|
CC benety.goh milkie |