[SERVER-8579] Consolidate Mongod Lock/Resource Scheduling Logic Created: 14/Feb/13  Updated: 02/Aug/18  Resolved: 30/Oct/14

Status: Closed
Project: Core Server
Component/s: Concurrency
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Andy Schwerin Assignee: Kaloian Manassiev
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Gantt Dependency
has to be done before SERVER-1423 reads often aren't possible while in ... Closed
Related
related to SERVER-11670 attempt to create a large collection ... Closed
is related to SERVER-3531 map reduce doesn't seem to yield unle... Closed
is related to SERVER-5724 dbtemprelease's destructor can fail t... Closed
is related to SERVER-8161 locks leak Closed
is related to SERVER-8378 indexbg1.js failures Closed
is related to SERVER-9147 touch command does not yield Closed
is related to SERVER-3931 Shell hangs on tab complete if db is ... Closed
is related to SERVER-8342 Delete old query_yield tests Closed
is related to SERVER-1256 low priority write flag Closed
is related to SERVER-1423 reads often aren't possible while in ... Closed
is related to SERVER-3642 make sure yieldSometimes RecordNeeds ... Closed
is related to SERVER-6183 count may not yield before paging in ... Closed
is related to SERVER-8654 yielding too much? Closed
is related to SERVER-3801 Mapreduce should yield periodically w... Closed
is related to SERVER-8307 Use db-level lock when MapReduce inpu... Closed
is related to SERVER-3456 Running fsync+lock on config server d... Closed
Participants:

 Description   

A unified scheduling module would consolidate logic around what operations get dispatched in parallel with other operations (fsync+lock and reads, e.g.), could ultimately manage resources (limit iops per individual user op, or disk space dedicated to external sorts at once), and more.

TODO: Fill out the description w/ a fuller design proposal.



 Comments   
Comment by Andy Schwerin [ 07/Jan/14 ]

Agree, needs more specification. fsync+lock, finer granularity, like you said. The requirements are things like "critical read operations shouldn't get stuck behind undispatched writes", where critical read operations might be things like commands involved in replicaset elections. This ticket is initially about (1), but (1) and (2) are more the same than different, I think. A lock on the right to do an external sort would allow you to use one set of scheduling logic for both, i.e.

Comment by Dwight Merriman [ 07/Jan/14 ]

does this ticket refer to:
(1) a replacement for mongo::Lock (d_concurrency.h) that is better
or
(2) a high level scheduler that isn't so much about locking but truly about scheduling. limiting # of ext sorts at the same time i think of as that.

is the goal here to do #1 or #2? of course there is some overlap, but i could imagine even rewritten those being separate abstractions or notions.

Comment by Dwight Merriman [ 07/Jan/14 ]

i think the first thing we need here is the requirements specification. i'm not clear what all is needed beyond solving fsync+lock.

i suppose more hierarchy, so locks can be more granular

what else?

Generated at Thu Feb 08 03:17:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.