[SERVER-41020] Tweak or fuzz storage engine tunable parameters in testing Created: 06/May/19  Updated: 29/Oct/23  Resolved: 06/Nov/20

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Improvement Priority: Major - P3
Reporter: Louis Williams Assignee: Brian DeLeonardis (Inactive)
Resolution: Fixed Votes: 2
Labels: execution_intern, neweng, newgrad
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File fuzz_configuration.py    
Issue Links:
Related
related to SERVER-51598 Add new test suites that test transac... Closed
is related to SERVER-41492 Disable WiredTiger cursor caching and... Closed
is related to SERVER-41494 Introduce more aggressive WiredTiger ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-09-21, Execution Team 2020-10-05, Execution Team 2020-10-19, Execution Team 2020-11-02, Execution Team 2020-11-16
Participants:

 Description   

This ticket tracks a list of tunable parameters in both MongoDB and WiredTiger that can be changed or fuzzed during testing to increase coverage.

Here's a start:

  • setParameter wiredTigerCursorCacheSize. We should run tests when this is set to 0 to disable cursor caching.
  • setParameter wiredTigerSessionCloseIdleTimeSecs. This is already set to 5 sec in debug builds.
  • setParameter wiredTigerConcurrentWrite/ReadTransactions. We already lower this to 64 in certain test suites.
  • WiredTiger file_manager parameters to wiredtiger_open (i.e. close_handle_minimum, close_idle_time, and close_scan_interval). This would introduce more aggressive file handle sweeps.
  • WiredTiger cache eviction parameters: eviction_target, eviction_trigger, eviction_dirty_target, eviction_dirty_trigger. These parameters are a function of the cache size which is already pretty low in most of our tests (1GB), so tweaking these may not be as effective as other options.


 Comments   
Comment by Githook User [ 19/Nov/20 ]

Author:

{'name': 'Brian DeLeonardis', 'email': 'brian.deleonardis@mongodb.com', 'username': 'bdeleonardis1'}

Message: SERVER-41020 Ensure that config fuzzer configurations are reproducible
Branch: master
https://github.com/mongodb/mongo/commit/9e258a74a0a3bfe725354cad653991aac7606e2b

Comment by Githook User [ 06/Nov/20 ]

Author:

{'name': 'Brian DeLeonardis', 'email': 'brian.deleonardis@mongodb.com', 'username': 'bdeleonardis1'}

Message: SERVER-41020 Add tests that fuzz storage engine parameters
Branch: master
https://github.com/mongodb/mongo/commit/a53d94e4849afcce37f7922616a733c862ac80a5

Comment by Brian DeLeonardis (Inactive) [ 22/Oct/20 ]

Code review url: https://mongodbcr.appspot.com/700950001/

Comment by Daniel Gottlieb (Inactive) [ 23/Apr/20 ]

Other interesting parameters that may make sense to fuzz:

Comment by Daniel Gottlieb (Inactive) [ 23/Apr/20 ]

I attached a POC python program that can output a string mongod CLI string:

$ python ../fuzz_configuration.py 
--setParameter enableTestCommands=1 --setParameter flowControlMinTicketsPerSecond=100 --setParameter flowControlThresholdLagPercentage=0.5 --setParameter wiredTigerSessionCloseIdleTimeSecs=100 --setParameter wiredTigerConcurrentWriteTransactions=26734 --setParameter flowControlTargetLagSeconds=10 --setParameter flowControlSamplePeriod=1000 --setParameter wiredTigerConcurrentReadTransactions=33654 --setParameter enableFlowControl=true --setParameter flowControlMaxSamples=10000 --setParameter wiredTigerCursorCacheSize=-917 --setParameter failpoint.flowControlTicketOverride={mode:'alwaysOn',data:{'numTickets': 974794}} --wiredTigerEngineConfigString=eviction_checkpoint_target=32,eviction_dirty_target=55,eviction_dirty_trigger=51,eviction_target=65,eviction_trigger=71,file_manager=(close_handle_minimum=165,close_idle_time=240,close_scan_interval=78)

That hopefully demonstrates that developing a fuzzed configuration should be simple (even if we start from scratch). It may make sense for the output to be a yaml file. The other hurdle is hooking it up appropriately to resmoke (which I've also POCed, very small change) and evergreen (probably the most challenging).

Comment by Eric Milkie [ 04/Jun/19 ]

Louis and I discussed this work, and we decided to create two new specific work tickets (linked) for the most valuable testing changes we can make at this time. I recommend we schedule those two tickets as part of this epic and move this ticket out of the Epic as a longer-term desire.

Comment by Daniel Gottlieb (Inactive) [ 21/May/19 ]

While this ticket was originally intended to just fuzz some WT tunables, it may be worthwhile considering some other values to tweak such as flow control.

Generated at Thu Feb 08 04:56:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.