[SERVER-76344] Support multiversion testing with the oldest binaries up through the latest binaries Created: 20/Apr/23  Updated: 26/May/23  Resolved: 26/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Dianna Hohensee (Inactive)
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-75922 Partial unique indexes created on Mon... Closed
related to SERVER-74891 Change binary download urls to accomm... Closed
is related to SERVER-77524 Explore unit testing 4.0 partial inde... Closed
Assigned Teams:
Server Development Platform
Sprint: Execution Team 2023-05-29
Participants:

 Description   

I would like to add a test that starts in v4.0 and upgrades through master (v7.1). This would allow testing of SERVER-75922, an unique partial index bug fix.

More generally this would allow continuing testing of server format or behavior changes throughout many version whenever it is important to ensure support.

Maybe someday we could even create some sort of passthrough suite that randomly upgrades/downgrades binaries.



 Comments   
Comment by Dianna Hohensee (Inactive) [ 26/May/23 ]

I'm closing this ticket out as Won't Do. Supporting 4.0 binaries forever into the distant future isn't sustainable. I'm pivoting to attempting to create a unit test that inserts the old 4.0 index data format: SERVER-77524.

If SERVER-77524 doesn't pan out, the final recourse will likely be something like setting up a 4.0 server with the repro data, upgrading it to a version we want to test, and then placing the server data files into a Cloud Bucket to provision onto test machines where a JS test can restore a mongod on top of the data files and run the test to ensure compatibility. This would require some manual maintenance to pull down the data files to run upgrade to the latest version and then stash the new data files for new version testing.

Comment by Dianna Hohensee (Inactive) [ 22/May/23 ]

It looks like I can run that code successfully in 7.0, however: here's the patch build – the redness in my test is from after upgrading from 4.0 to 7.0 successfully.

Comment by Dianna Hohensee (Inactive) [ 22/May/23 ]

It looks like there was 6.2 work to remove support for 4.0 binaries (SERVER-71097) and 7.1 work to remove support for 4.2 binaries (SERVER-75943). I'm getting setup errors in my patch build about not of build variant support for a version.

[2023/05/22 20:28:39.954] [2023-05-22 20:28:39,954 - db_contrib_tool.setup_repro_env.setup_repro_env - INFO] 2023-05-22 20:28.39 [info     ] Setting up request             request=RequestTarget(request_type=<RequestType.MONGO_RELEASE_VERSION: 'mongo_release_version'>, identifier='4.0')
[2023/05/22 20:28:39.954] [2023-05-22 20:28:39,954 - db_contrib_tool.setup_repro_env.setup_repro_env - INFO] 2023-05-22 20:28.39 [info     ] Fetching download URLs from Evergreen
[2023/05/22 20:28:40.025] [2023-05-22 20:28:40,025 - db_contrib_tool.setup_repro_env.artifact_discovery_service - DEBUG] 2023-05-22 20:28.40 [debug    ] Found evergreen project        evergreen_project=mongodb-mongo-v4.0
[2023/05/22 20:28:40.025] [2023-05-22 20:28:40,025 - db_contrib_tool.setup_repro_env.artifact_discovery_service - DEBUG] 2023-05-22 20:28.40 [debug    ] Searching for build variant    target=DownloadTarget(edition='enterprise', platform='amazon2', architecture='aarch64') version=4.0
[2023/05/22 20:28:40.025] [2023-05-22 20:28:40,025 - db_contrib_tool.setup_repro_env.artifact_discovery_service - DEBUG] 2023-05-22 20:28.40 [debug    ] Found buildvariant             buildvariant_name=enterprise-amazon2-arm64
[2023/05/22 20:28:40.027] Traceback (most recent call last):
[2023/05/22 20:28:40.027]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/bin/db-contrib-tool", line 8, in <module>
[2023/05/22 20:28:40.027]     sys.exit(cli())
[2023/05/22 20:28:40.027]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
[2023/05/22 20:28:40.027]     return self.main(*args, **kwargs)
[2023/05/22 20:28:40.027]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/core.py", line 1055, in main
[2023/05/22 20:28:40.027]     rv = self.invoke(ctx)
[2023/05/22 20:28:40.027]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
[2023/05/22 20:28:40.028]     return _process_result(sub_ctx.command.invoke(sub_ctx))
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/usage_analytics.py", line 61, in invoke
[2023/05/22 20:28:40.028]     return super().invoke(ctx)
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
[2023/05/22 20:28:40.028]     return ctx.invoke(self.callback, **ctx.params)
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/core.py", line 760, in invoke
[2023/05/22 20:28:40.028]     return __callback(*args, **kwargs)
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
[2023/05/22 20:28:40.028]     return f(get_current_context(), *args, **kwargs)
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/setup_repro_env/cli.py", line 269, in setup_repro_env
[2023/05/22 20:28:40.028]     success = setup_repro_orchestrator.execute(setup_repro_params)
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/setup_repro_env/setup_repro_env.py", line 219, in execute
[2023/05/22 20:28:40.028]     evg_urls_info = self.artifact_discovery_service.find_artifacts(
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/setup_repro_env/artifact_discovery_service.py", line 144, in find_artifacts
[2023/05/22 20:28:40.028]     artifact_urls = self.scan_for_artifact_urls(
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/setup_repro_env/artifact_discovery_service.py", line 292, in scan_for_artifact_urls
[2023/05/22 20:28:40.028]     urls = self.find_artifact_urls_for_version(
[2023/05/22 20:28:40.028]   File "/data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/setup_repro_env/artifact_discovery_service.py", line 248, in find_artifact_urls_for_version
[2023/05/22 20:28:40.028]     raise MissingBuildVariantError(
[2023/05/22 20:28:40.028] db_contrib_tool.setup_repro_env.artifact_discovery_service.MissingBuildVariantError: Buildvariant 'enterprise-amazon2-arm64' not found in evergreen. Available buildvariants can be found in /data/mci/a32b91f542d939e1ba5c20d13e53ea9f/pipx/venvs/db-contrib-tool/lib/python3.10/site-packages/db_contrib_tool/config/setup_repro_env_config.yml.

Comment by Alex Neben [ 01/May/23 ]

Hey I just wanted to follow up on this. We think from our side the infra is there (or 99% there if you need to revert some stuff). However, imho we should not be testing EoL binaries since they are not going to work on modern OSes (for example openssl 3 vs 1.1). I think another option is to create the index or the data files you need and ask the build team to upload them somewhere. Then you can download them and use them in your test. Either way I think this is a test writing specific issue which should be done by individual teams.

Comment by Tausif Rahman (Inactive) [ 27/Apr/23 ]

FYI, max.hirschhorn@mongodb.com pointed out that there may be some special handling needed when downloading 4.0 binaries. Check this. I don't recall exactly what that was but I think it may have to do with certain platforms in CI not having 4.0 binaries available. You may need to tweak some things for this new test to run successfully on all platforms.

Comment by Tausif Rahman (Inactive) [ 27/Apr/23 ]

The multiversion suite actually runs tests that rely on older versions of binaries. Here's an example. I think you should be able to use the existing infrastructure. The setup multiversion script (which downloads older versions of mongodb for multiversion testing) only goes back to 4.4 though. If you need older versions, you can try adding 4.2/4.0, but I don't recall if those binaries are available via db-contrib-tool since those are EOL.

Generated at Thu Feb 08 06:32:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.