[SERVER-73998] Fix system_perf.yml and perf.yml microbench buildvariant definitions Created: 14/Feb/23  Updated: 29/Oct/23  Resolved: 17/Feb/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0, 6.0.5, 6.3.0-rc1

Type: Bug Priority: Blocker - P1
Reporter: James O'Leary Assignee: Jawwad Asghar
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.3, v6.0
Participants:
Linked BF Score: 135

 Description   

Evergreen has upgraded it's yaml library to be more correct / consistent.

As a result, some of the microbenchmarks build variants are broken.

We need to fix the broken variants to be of the following form:

- &linux-microbenchmarks-standalone-arm
  ... stuff ...
  expansions: &new-alias-name-for-expansions
 
...other configuration...
- <<: *linux-microbenchmarks-standalone-arm
  ...other stuff...
  expansions:
    <<: *new-alias-name-for-expansions
    mongodb_setup: microbenchmarks_standalone-all-feature-flags.2023-01

These changes may need to be applied to both the performance and system performance projects.



 Comments   
Comment by Githook User [ 17/Feb/23 ]

Author:

{'name': 'Jawwad Asghar', 'email': 'jawwad.asghar@mongodb.com', 'username': 'jawwadasghar'}

Message: SERVER-73998 Resolving MicroBenchmark issue with not all tasks running on certain variants
Branch: master
https://github.com/mongodb/mongo/commit/4702824a2cb37317a4af7842f1af48736f0fd08d

Comment by Jawwad Asghar [ 16/Feb/23 ]

Standup:

  • Shorty after we submitted this fix to address the evergreen change in how expansions are handled, this[ SERVER-72471|https://github.com/10gen/mongo/commit/f4bcbe1635ba16663bca6d53f73d5146ba3b5eeb] change was made adding a task to 2 microbenchmark variants. One would assume that the added task would be merged with the task list defined in the referenced anchor variant, but Yaml cannot merge arrays nicely (see [this yaml git issue|https://github.com/yaml/yaml/issues/35] ), so the change caused only the uniquely defined task to run for those variants!
  • I confirmed with alexander.ignatyev@mongodb.com that we do indeed only want the newly added task to run for a subset of microbenchmark standalone variants (only want to run it on all-feature-flags-enabled variants)
  • I created this pull request where we define a new task list for the diverging variant, and also added some comments in to hopefully prevent something like this from happening in the future. 
  • I dont see any backports for the SERVER-72471 change that added these 2 tasks, so I dont think a divergent task list is necessary in our older branches for now, though we might want to add the new comments from this pull there for the sake of resiliency 

FYI jim.oleary@mongodb.com 

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'Jawwad Asghar', 'email': 'jawwad.asghar@mongodb.com', 'username': 'jawwadasghar'}

Message: SERVER-73998 Fixing expansions issue with sys-perf microbenchmark tests

(cherry picked from commit c0c1e7dff1a50765458e9f22f37e5f543ac0bc16)
Branch: v6.3
https://github.com/mongodb/mongo/commit/a0ee5a1e6ce59b4794b066fdb6863eaba7b879dd

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'dalyd', 'email': 'david.daly@mongodb.com', 'username': 'dalyd'}

Message: SERVER-73998: Fix perf.yml

(cherry picked from commit 638f348aea94c7fa0d26ffcaeb4380430a54d5e6)
Branch: v6.0
https://github.com/mongodb/mongo/commit/4213cf5b6c7353a7194383f4ad70f55dd84aa852

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'dalyd', 'email': 'david.daly@mongodb.com', 'username': 'dalyd'}

Message: SERVER-73998: Fix perf.yml

(cherry picked from commit 638f348aea94c7fa0d26ffcaeb4380430a54d5e6)
Branch: v6.3
https://github.com/mongodb/mongo/commit/099b1a174e90178253af7281d9ef98cad45e947d

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'dalyd', 'email': 'david.daly@mongodb.com', 'username': 'dalyd'}

Message: SERVER-73998: Fix perf.yml

(cherry picked from commit 638f348aea94c7fa0d26ffcaeb4380430a54d5e6)
Branch: BACKPORT-14790-perf
https://github.com/mongodb/mongo/commit/c1547bc49a86738cde8e0a920290d373ed3dc242

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'Jawwad Asghar', 'email': 'jawwad.asghar@mongodb.com', 'username': 'jawwadasghar'}

Message: SERVER-73998 Fixing expansions issue with sys-perf microbenchmark tests

(cherry picked from commit c0c1e7dff1a50765458e9f22f37e5f543ac0bc16)
Branch: BACKPORT-14789-sys-perf
https://github.com/mongodb/mongo/commit/31895f53cd495e1e7b3840c1ff544acdb9ee3bd0

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'dalyd', 'email': 'david.daly@mongodb.com', 'username': 'dalyd'}

Message: SERVER-73998: Fix perf.yml

(cherry picked from commit 638f348aea94c7fa0d26ffcaeb4380430a54d5e6)
Branch: BACKPORT-14789-perf
https://github.com/mongodb/mongo/commit/696149f9a0b12c4f20b5f35f5f5155673ba2f74a

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'Jawwad Asghar', 'email': 'jawwad.asghar@mongodb.com', 'username': 'jawwadasghar'}

Message: SERVER-73998 Fixing expansions issue with sys-perf microbenchmark tests
Branch: master
https://github.com/mongodb/mongo/commit/c0c1e7dff1a50765458e9f22f37e5f543ac0bc16

Comment by Githook User [ 15/Feb/23 ]

Author:

{'name': 'dalyd', 'email': 'david.daly@mongodb.com', 'username': 'dalyd'}

Message: SERVER-73998: Fix perf.yml
Branch: master
https://github.com/mongodb/mongo/commit/638f348aea94c7fa0d26ffcaeb4380430a54d5e6

Comment by James O'Leary [ 14/Feb/23 ]

It looks like the perf.yml file is also broken (probably in a more significant way that system_perf.yml).

Comment by Xiaochen Wu [ 14/Feb/23 ]

This should be P1. Can we fix this asap? 

Generated at Thu Feb 08 06:26:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.