[SERVER-32997] Mobile SE: Design and implement multi-reader or single-writer concurrency Created: 30/Jan/18 Updated: 29/Oct/23 Resolved: 18/Jun/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.1, 4.1.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Sulabh Mahajan | Assignee: | Sulabh Mahajan |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | SERG, nonnyc, storage-engines | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||
| Backport Requested: |
v4.0
|
||||||||||||||||||||||||||||
| Sprint: | Storage Non-NYC 2018-05-07, Storage Non-NYC 2018-05-21, Storage Non-NYC 2018-06-04, Storage Non-NYC 2018-06-18 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Linked BF Score: | 26 | ||||||||||||||||||||||||||||
| Story Points: | 13 | ||||||||||||||||||||||||||||
| Description |
|
|
| Comments |
| Comment by Githook User [ 05/Jul/18 ] |
|
Author: {'username': 'sulabhM', 'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com'}Message: (cherry picked from commit cc8aab5249ccdec471e33bda087faecb53a6d9bf) |
| Comment by Githook User [ 18/Jun/18 ] |
|
Author: {'username': 'sulabhM', 'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com'}Message: |
| Comment by Sulabh Mahajan [ 22/May/18 ] |
|
I had a discussion with milkie regarding the above approaches and we concluded that it is the best to modify lock manager to support instance level locking for mobile SE. There are places where proper locks are not held when doing reads or writes, The plan proposed for now is to put debug code in the mobile SE to identify places where proper locks from instance manager are not taken. I ran a few tests and I think a few of these issues are in the code related to capped collections, rename collection and validate collections. I am still debugging to get more details. |
| Comment by Eric Milkie [ 16/May/18 ] |
|
How is approach 2 different from using instance-level-locking? Presumably, all the places where we have a WriteUnitOfWork, we have already acquired a Global IX lock, which for instance-level-locking can be a Global X lock instead. Then you would achieve the same effect as doing internal exclusive locking for WUOW's. |
| Comment by Sulabh Mahajan [ 16/May/18 ] |
|
The patch making fixes for the mobile SE didn't prove to be useful. The test run showed additional crashes other than the existing locked/busy issues. We tried couple of other approaches:
|
| Comment by Sulabh Mahajan [ 09/May/18 ] |
|
milkie suggested to try the attached patch which implements supportsDBLocking for the mobile SE. This is along the lines of the 1st approach as suggested above. This fixes/enhances the lock manager to provide the total instance locking by converting Global IX and IS locking into X and S locks, respectively.
We have a meeting setup to continue discussing the solution for this issue. I will put an update as we make progress. |
| Comment by Sulabh Mahajan [ 04/May/18 ] |
|
Root Cause: This is caused by a mismatch in how concurrency is handled in MongoDB vs SQLite:
Possible Solutions:
After a discussion within the team we have decided to explore the 2nd approach above. This will localise the concurrency control inside the SQLite SE alone, making it a simpler (to implement and debug) approach than the other. Also there is a desire to get away from having to support storage engines that advertise themselves being database wide concurrent. |