[SERVER-39859] Use the Evergreen REST v2 API to download -latest tarballs in setup multiversion mongodb Created: 27/Feb/19 Updated: 29/Oct/23 Resolved: 08/Dec/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.9.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Max Hirschhorn | Assignee: | Mikhail Shchatko |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | neweng, tig-multiversion, tig-qwin-eligible | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.4
|
||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | STM 2020-11-30, STM 2020-12-14 | ||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 15 | ||||||||||||||||||||||||||||||||||||||||||||
| Story Points: | 2 | ||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
There are caching policies for fastdl.mongodb.org and downloads.mongodb.com that cause --useLatest to download day-old
An unfortunate consequence of using Evergreen to download binaries from older branches is that we would tie build variants across different mongodb-mongo-* Evergreen projects together when we haven't necessarily been consistent about naming. This means it won't always be possible to download the binaries from the build variant with the same name as the one that exists in mongodb-mongo-master and we may need to keep an explicit mapping in etc/evergreen.yml. We must also make a decision about whether we should depend on the push task succeeding, or only depend on compile, jsCore, dbtest, and replica_sets_jscore_passthrough succeeding as |
| Comments |
| Comment by Githook User [ 08/Dec/20 ] | |||||||
|
Author: {'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com'}Message: | |||||||
| Comment by Githook User [ 08/Dec/20 ] | |||||||
|
Author: {'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com'}Message: | |||||||
| Comment by Brooke Miller [ 02/Nov/20 ] | |||||||
|
As we discussed in triaging, given
| |||||||
| Comment by Mikhail Shchatko [ 21/Oct/20 ] | |||||||
|
Picking out `map the current args to evergreen build variants` part into | |||||||
| Comment by Robert Guo (Inactive) [ 15/Oct/20 ] | |||||||
|
We're going to break this ticket up into multiple pieces. 1. auth to the Evergreen API and S3 for both local and Evergreen machines | |||||||
| Comment by Max Hirschhorn [ 29/Apr/20 ] | |||||||
This came up with Blake and Marcos today when they were looking at a multiversion failure. I realized because the -latest tarballs come directly from the mongodb-mongo-v* Evergreen projects (i.e. the binaries aren't rebuilt from scratch like we do for releases) that it is very possible to get the corresponding binaries and debug symbols provided the logs are still available. 1. Search for the start-up message which says "git version" or "gitVersion".
2. Manually construct the /version Evergreen URL https://evergreen.mongodb.com/version/mongodb_mongo_v4.4_e33cb0c81df6901645788f1df484c2d492c08b1f using the MongoDB version and githash. 3. Click on the compile task for the appropriate build variant (e.g. Enterprise RHEL 6.2 and https://evergreen.mongodb.com/task/mongodb_mongo_v4.4_enterprise_rhel_62_64_bit_compile_e33cb0c81df6901645788f1df484c2d492c08b1f_20_04_28_20_29_34). This step may be the most challenging because it requires interpreting the modules, distmod, platform, and architecture settings of the binary in the log output. 4. Confirm the build-id of the executable.
| |||||||
| Comment by Eric Milkie [ 02/Mar/20 ] | |||||||
|
This work could be useful as it would mean that we could also download the debug symbols for these binaries from Evergreen. Currently, if one of the older binaries crashes and leaves a core dump, we have no way of using it, since the binary and debug symbols are not saved. | |||||||
| Comment by Max Hirschhorn [ 27/Feb/19 ] | |||||||
|
The "or possibly month-old binaries" was inaccurate and based on some confusion around the author date vs commit date. I think the worst we've seen is between a few hours and 1 day. |