[SERVER-70879] Fix race where multiple threads are turning an index multikey concurrently Created: 26/Oct/22 Updated: 29/Oct/23 Resolved: 04/Nov/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 6.1.1, 5.0.14, 6.0.3, 6.2.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Fausto Leyva (Inactive) | Assignee: | Henrik Edin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Backport Requested: |
v6.1, v6.0, v5.0
|
||||||||
| Sprint: | Execution Team 2022-11-14 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 115 | ||||||||
| Description |
|
In BF-26686 we see an inconsistent multikey state in the in-memory catalog.
Notably, the first insert will set the multikey path for a and b while the second document will only set b. Ultimately, we only have b as a multikey path (on disk). We potentially have a situation where writers could be obtaining copies of the collection metadata before it is marked as multikey and the second writer could overwrite the other writer’s collection metadata write. This ticket would include writing a detailed c++ test to verify the findings and potential bug. |
| Comments |
| Comment by Githook User [ 09/Nov/22 ] |
|
Author: {'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}Message: Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer. To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable. (cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392) |
| Comment by Githook User [ 08/Nov/22 ] |
|
Author: {'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}Message: Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer. To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable. (cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392) |
| Comment by Githook User [ 08/Nov/22 ] |
|
Author: {'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}Message: Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer. To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable. (cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392) |
| Comment by Githook User [ 04/Nov/22 ] |
|
Author: {'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}Message: Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer. To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable. |