[SERVER-5729] Special-case concurrency model for oplog, make reads not block as much Created: 30/Apr/12  Updated: 06/Dec/22  Resolved: 14/Sep/18

Status: Closed
Project: Core Server
Component/s: MMAPv1, Replication, Storage
Affects Version/s: 2.0.4
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Jared D. Cottrell Assignee: Backlog - Storage Execution Team
Resolution: Done Votes: 4
Labels: sync
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Ubuntu 10.10 64-bit


Issue Links:
Related
related to SERVER-20328 Allow secondary reads while applying ... Closed
related to SERVER-21862 Use record store directly to read fro... Closed
Assigned Teams:
Storage Execution
Participants:

 Description   

It would be useful to special-case the concurrency model for the oplog so that reads don't block. Under sustained periods of high write load, secondaries can fall further and further behind the primary because of high lock% when trying to read from the oplog. But the oplog is a special beast that is an append-only (sorta, most of the time) capped collection. It might be possible to use this fact to make special locking rules that allowed reads to happen all the time, or at least dramatically reduce the period of lock contention.

The situation will get markedly better with DB-level locking, but further special-casing could help as well. Currently we have to bump the horsepower on the nodes in the replica set, even if the boxes are otherwise healthy.

There are sometimes reasons to actually write to the oplog manually. Would need to figure out a way to deal with those cases.



 Comments   
Comment by Asya Kamsky [ 21/Dec/17 ]

Probably can be closed as duplicate of SERVER-20328 since implementing that ticket will resolve this one.

Comment by Andy Schwerin [ 30/Apr/12 ]

I'm labeling this "features we're not sure of", because we don't know, yet, if a special-case approach to oplog access will be appropriate. That will depend a lot on how other concurrency improvements improve or stymy oplog data distribution.

Comment by Andy Schwerin [ 30/Apr/12 ]

It's an interesting thought. Since the oplog is in a distinct database ("local"), which is only write-locked for a brief period at the end of document write operations, we're expecting distinctly improved secondary performance under high write load. Further down the road, finer grained locking (i.e., sub collection level), or other concurrency techniques should improve read access to all databases under heavy write load, and raise the performance of replication along with it.

Once db-level locking is in place, I think a bigger problem will be keeping secondaries caught up applying the oplog, rather than getting the oplog data off the primaries. The v2.0 approach to oplog application is inherently single-threaded, and that's not going to suffice in the world of db-level locking.

Generated at Thu Feb 08 03:09:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.