[SERVER-42226] Use the maxValidateMBperSec server parameter to throttle collection validation operations Created: 12/Jul/19  Updated: 29/Oct/23  Resolved: 23/Aug/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.3.1

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-42224 Add a server parameter maxValidateMBP... Closed
is depended on by SERVER-42357 Periodically release locks during col... Closed
Problem/Incident
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2019-08-26
Participants:
Linked BF Score: 0

 Description   

SERVER-42224 will have added maxValidateMBperSec as a server parameter and set it up as accessible to the validation command code path.

This task is to actually start using maxValidateMBperSec to throttle the progress of a collection validation operation. Throttling should only occur when {background:true} is requested. We should put the thread to sleep after it processes maxValidateMBperSec MBs worth of data, if it take less than a second to reach maxValidateMBperSec MBs. We should check for interrupt after sleeping.



 Comments   
Comment by Gregory Wlodarek [ 23/Aug/19 ]

After discussing the unit test flakiness on Windows machines regarding timing with milkie, we're going to disable the throttle cursor unit test temporarily and file a ticket to fix the issues and to turn it back on.

Comment by Githook User [ 23/Aug/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-42226 Disable the throttle cursor unittest
Branch: master
https://github.com/mongodb/mongo/commit/2fd30c80a014077844e6a4ddb926e3ebef4868a0

Comment by Githook User [ 23/Aug/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-42226 Use the maxValidateMBperSec server parameter to throttle collection validation operations
Branch: master
https://github.com/mongodb/mongo/commit/46522393008ad96af478aa1dd5122ea6e4960889

Comment by Githook User [ 23/Aug/19 ]

Author:

{'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'name': 'Gregory Wlodarek'}

Message: SERVER-42226 Clarify that the 'maxValidateMBperSec' server parameter is only used when the validate command is running in the background.
Branch: master
https://github.com/mongodb/mongo/commit/98ae88a36c731e79ed1a299b954c0dfafd7b096c

Comment by Githook User [ 23/Aug/19 ]

Author:

{'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'name': 'Gregory Wlodarek'}

Message: SERVER-42226 Add a data throttle mechanism for cursors to use that allows them to specify the maximum throughput per second in order to limit I/O and CPU usage.

Additionally, cursor wrappers for the validate command have been implemented to use the data throttle mechanism.
Branch: master
https://github.com/mongodb/mongo/commit/8f34015c725e56eb5550c7ae42010de85ad3dbf2

Generated at Thu Feb 08 04:59:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.