[SERVER-27823] Update microbenchmarks to run against wiredTiger in addition to inMemory Created: 15/Dec/16  Updated: 04/Dec/23  Resolved: 04/Dec/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.1 Desired

Type: Task Priority: Major - P3
Reporter: David Daly Assignee: Backlog - Storage Execution Team
Resolution: Won't Do Votes: 2
Labels: mtig, perf-stop-regressions, quick_wins
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-26685 Primary journal and oplog visibility ... Closed
related to SERVER-21316 Add inMemory Storage engine build var... Closed
is related to SERVER-26981 Update mongo-perf Closed
Assigned Teams:
Storage Execution
Participants:
Linked BF Score: 0

 Description   

1. We would use the same command line arguments as before:

mongod_flags: "--storageEngine=wiredTiger --logpath ./mongod.log --fork --syncdelay 0 --nojournal --setParameter ttlMonitorEnabled=false --setParameter diagnosticDataCollectionEnabled=false --wiredTigerCacheSizeGB 16"

2. We would only run the standalone variant for wiredTiger, that means without oplog. This configuration does not suffer from the issue with fsyncs being necessary for commits, which was the reason to move to inMemory engine.

2b. We would not run wiredTiger with oplog on at all in Microbenchmarks. The Microbenchmarks project is specifically designed to NOT do IO, and having one variant that does that would just be confusing and we would probably waste a lot of time trying to understand and reproduce the results. (On this note: hopefully we get sys-perf back in the game soon.)

3. Unrelated, but the current display names are a bit sub-optimal in the UI. Having the storage engine last, it is often not shown at all in the UI. The word Linux is redundant, as that's constant. So I would rather switch to "Standalone wiredTiger Linux" (with Linux last)

3b. For same reasons also proposing to change "1-Node ReplSet" to just "ReplSet1"

4. When we changed to inMemory engine, we didn't change the Evergreen variant ids, because that would have been very hard (we wanted to keep the history). We only changed the display name. So now we have:

buildvariants:
  - name: linux-wt-standalone
    display_name: Standalone Linux inMemory

If we want to get wiredTiger engine back running, I would suggest we abandon this hack, otherwise it becomes too confusing. The variants would then be:

  - name: linux-wt-standalone
    display_name: Standalone wiredTiger Linux
    ...
  - name: linux-memory-standalone
    display_name: Standalone inMemory Linux
    ...
  - name: linux-memory-repl
    display_name: ReplSet1 inMemory Linux
    ...

5. For clarity:

  • linux-wt-standalone would regain the history of Standalone wiredTiger tests, with a short period of inMemory engine in between
  • What is currently linux-wt-repl is discontinued and history is not inherited by anyone
  • linux-memory-standalone and linux-memory-repl will start fresh with no history


 Comments   
Comment by Steven Vannelli [ 04/Dec/23 ]

miguel.nieto@mongodb.com - discussed with the team and we're comfortable closing this for now. Thanks!

Comment by Alexander Gorrod [ 25/Jun/23 ]

This still seems relevant - I believe we should be running our benchmarking in an environment as similar to what our users run as possible - even micro benchmarks.
OTOH: This hasn't had traction for a long time, and I don't see it being a priority immediately, so maybe it's best to close this out rather than leave it languishing.

Comment by Miguel Angel Nieto [ 14/Jun/23 ]

Moved the ticket to Storage Execution Backlog. This is a 5 years old ticket to update microbenchmarks so they run on WiredTiger in addition to inMemory. Reaching out to you in order to get your opinion and understand if this is something you think it is still needed.

Comment by Henrik Ingo (Inactive) [ 31/May/18 ]

To discuss before implementing:

  • Double check with server engineer that the fsyncs introduced in SERVER-26685 are still happening and needed. I'm mostly thinking wrt moving to using mongo timestamps in the wt journal.
  • Nowadays we seem much more confident in copying history data than when this ticket was written. The description can probably be much simplified if going that route.
Generated at Thu Feb 08 04:16:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.