[SERVER-62992] Remove need for resmoke.ini Created: 26/Jan/22  Updated: 29/Oct/23  Resolved: 11/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.0-rc0, 6.0.0-rc2, 5.0.9, 4.4.15

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Richard Samuels (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
is duplicated by SERVER-59648 make resmoke.ini a build node Closed
Problem/Incident
causes SERVER-67398 The build with the legacy install mod... Closed
Related
is related to SERVER-63104 Make it easy to run jsCore locally ag... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.0, v5.0, v4.4
Sprint: Dev Platform 2022-03-07, Dev Platform 2022-03-21, Dev Platform 2022-04-04, Dev Platform 2022-04-18
Participants:
Linked BF Score: 0
Story Points: 3

 Description   

The resmoke.ini file was added during the rollout of hygienic builds to deal with the fact that while we were transitioning between hygienic builds and legacy style builds, we needed resmoke to be able to look in different places for the binaries under test. Having the build system create the resmoke.ini file for resmoke to consume and understand the placement of the intended build, was, at the time, an OK workaround.

The result, however, has been long-term troublesome:

  • When Ninja is in play, there is an inherent impossibility of correctly updating this file with multiple Ninja files in play, due to the issues identified in SERVER-53952. No amount of SCons hacking is going to give us a resmoke.ini in the root of the tree that is under simultaneous control of multiple Ninja files and behaves correctly. As a result, engineers hand edit the file (see SERVER-59648).
  • The presence of resmoke.ini breaks the encapsulation of hygienic builds, where all build outputs are to be entirely quarantined within a build directory. But the resmoke.ini file can't be brought into that environment, because that would undermine its one use.

Long term, this problem should be obviated by the self-testable installs effort, where each installation will have its own entry points which test the install. However, we aren't there yet.

We need to find a way to eliminate the need for resmoke.ini file entirely.

One simple way to do this may be for resmoke itself to gain some heuristics to identify the available builds. For instance, if we gave each build a well known file (or keyed from one of the existing ones, like the license file), then resmoke could start by running a find command from cwd. If it finds one matching installation directory, it assumes that that is the intended build to test. If it finds zero, it reports an error. If it finds more than one, it either prompts the user to select the build to test, or logs the identified builds and errors out with a message to re-run with the installDir flag set as appropriate, if the environment is not interactive.

Alternatively, we could just require that resmoke always be invoked with installDir.

I'm happy to investigate other options.



 Comments   
Comment by Githook User [ 25/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Remove need for resmoke.ini (v4.4)
Branch: v4.4
https://github.com/mongodb/mongo/commit/2db61e54f7e964e02fab5d3c89e04fa007c782d0

Comment by Githook User [ 21/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Remove need for resmoke.ini (v5.0)
Branch: v5.0
https://github.com/mongodb/mongo/commit/f517542b0aff150da32a795b5d9dc6a7fb0f54b9

Comment by Githook User [ 20/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Don't validate resmoke.ini and install_dir when using subcommands other than run

(cherry picked from commit a4aa57a45b9c730acc0a888d0b86a479423ffe87)
Branch: v6.0
https://github.com/mongodb/mongo/commit/adfd03d5eb008910ceabc5bd62aaa50da6aca1ef

Comment by Githook User [ 18/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Don't validate resmoke.ini and install_dir when using subcommands other than run
Branch: master
https://github.com/mongodb/mongo/commit/a4aa57a45b9c730acc0a888d0b86a479423ffe87

Comment by Githook User [ 11/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Fix error message when user has multiple testable installations and hasn't specified which one to use
Branch: master
https://github.com/mongodb/mongo/commit/fdd96b07d7d42965eca34d303c303fe5d7acf2cc

Comment by Githook User [ 08/Apr/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Remove need for resmoke.ini
Branch: master
https://github.com/mongodb/mongo/commit/0d9705653d904bed75b06f77eba2a5e9af2b3a7d

Comment by Githook User [ 30/Mar/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: Revert "SERVER-62992 Remove need for resmoke.ini"

This reverts commit d4206bdeb19f4c5862280c04cd6b831c9ec03a40.
Branch: master
https://github.com/mongodb/mongo/commit/2ab888c8fe45f88ce9509a3690e083ad5f35e418

Comment by Richard Samuels (Inactive) [ 30/Mar/22 ]

Due to breakages in sdam_json_test, this change is being reverted.

Comment by Githook User [ 30/Mar/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-62992 Remove need for resmoke.ini
Branch: master
https://github.com/mongodb/mongo/commit/d4206bdeb19f4c5862280c04cd6b831c9ec03a40

Comment by Iryna Zhuravlova [ 03/Mar/22 ]

As a part of this ticket, we should make it backportable to 5.0 and 4.4. and communicate the changes to the db-contributors.

Comment by Brooke Miller [ 01/Feb/22 ]

During triage, we discussed that we'll pursue the suggestion in your comment, acm, to align with the broader vision to push more of this logic to the build system. We hope this will take us one step closer to self-testable installs.

Comment by Andrew Morrow (Inactive) [ 26/Jan/22 ]

Another approach to this would be to have SCons/Ninja generate a file into each installation directory like $DESTDIR/$PREFIX_BINDIR/resmoke, which was a wrapper script which would know how to correctly invoke buildscripts/resmoke.py --installDir <self>. If you had multiple installation directories from multiple ninja files, then you would invoke resmoke like:

  • build/opt/install/bin/resmoke ...
  • build/asan/install/bin/resmoke ...

But there are a lot of tricky things about that:

  • The solution needs to be cross platform and produce a correct script or program on all of Linux, Windows, and macOS.
  • It isn't clear what AIB component the generated resmoke wrapper should adhere to.
  • We would need to ensure that the resmoke.ini file didn't become part of packages.

On the other hand, we would probably need to do the same thing for any sentinel file that we created.

Generated at Thu Feb 08 05:56:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.