[SERVER-5729] Special-case concurrency model for oplog, make reads not block as much Created: 30/Apr/12 Updated: 06/Dec/22 Resolved: 14/Sep/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | MMAPv1, Replication, Storage |
| Affects Version/s: | 2.0.4 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Jared D. Cottrell | Assignee: | Backlog - Storage Execution Team |
| Resolution: | Done | Votes: | 4 |
| Labels: | sync | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Ubuntu 10.10 64-bit |
||
| Issue Links: |
|
||||||||||||
| Assigned Teams: |
Storage Execution
|
||||||||||||
| Participants: | |||||||||||||
| Description |
|
It would be useful to special-case the concurrency model for the oplog so that reads don't block. Under sustained periods of high write load, secondaries can fall further and further behind the primary because of high lock% when trying to read from the oplog. But the oplog is a special beast that is an append-only (sorta, most of the time) capped collection. It might be possible to use this fact to make special locking rules that allowed reads to happen all the time, or at least dramatically reduce the period of lock contention. The situation will get markedly better with DB-level locking, but further special-casing could help as well. Currently we have to bump the horsepower on the nodes in the replica set, even if the boxes are otherwise healthy. There are sometimes reasons to actually write to the oplog manually. Would need to figure out a way to deal with those cases. |
| Comments |
| Comment by Asya Kamsky [ 21/Dec/17 ] |
|
Probably can be closed as duplicate of |
| Comment by Andy Schwerin [ 30/Apr/12 ] |
|
I'm labeling this "features we're not sure of", because we don't know, yet, if a special-case approach to oplog access will be appropriate. That will depend a lot on how other concurrency improvements improve or stymy oplog data distribution. |
| Comment by Andy Schwerin [ 30/Apr/12 ] |
|
It's an interesting thought. Since the oplog is in a distinct database ("local"), which is only write-locked for a brief period at the end of document write operations, we're expecting distinctly improved secondary performance under high write load. Further down the road, finer grained locking (i.e., sub collection level), or other concurrency techniques should improve read access to all databases under heavy write load, and raise the performance of replication along with it. Once db-level locking is in place, I think a bigger problem will be keeping secondaries caught up applying the oplog, rather than getting the oplog data off the primaries. The v2.0 approach to oplog application is inherently single-threaded, and that's not going to suffice in the world of db-level locking. |