[SERVER-54086] Download debug symbols in resmoke hang analyzer Created: 27/Jan/21  Updated: 29/Oct/23  Resolved: 21/Apr/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 5.0.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Siran Wang (Inactive)
Resolution: Fixed Votes: 0
Labels: tig-hanganalyzer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-44483 Debug symbols archive should only be ... Closed
Gantt Dependency
has to be done before SERVER-56289 All generated tasks depend on archive... Closed
Related
related to SERVER-54297 Split up archiving of dist-test and d... Closed
Backwards Compatibility: Fully Compatible
Sprint: STM 2021-03-22, STM 2021-04-05, STM 2021-04-19, STM 2021-05-03
Participants:
Story Points: 3

 Description   

Currently we spend about 6 min wall clock time in every Evergreen build waiting for debug symbols to be archived and blocking execution of all JS tasks.

The archives are only used if the task fails, which is relatively rare. We should therefore conditionally wait for debug symbols only when a resmoke task fails or times out.

Impl sketch:

  1. Update multiversion setup to support downloading from patch builds
    1. When resmoke tries to run the hang analyzer, it should first check if the debug symbols are available and if not, retry downloading the debug symbols for up to 15 minutes since resmoke is first started. This only needs to be done if resmoke is running in Evergreen and it detects running mongod, mongos, or mongo processes. Some tests are run through resmoke but don't depend on compile or debug symbols.
    2. The resmoke start time needs to be persisted to disk like what Genny does since the hang analyzer is called on a separate resmoke instance.


 Comments   
Comment by Robert Guo (Inactive) [ 22/Apr/21 ]

Thanks for calling that out Drew... it was indeed not in the ticket description; I filed SERVER-56289 for changes to task dependencies.

Comment by Andrew Morrow (Inactive) [ 22/Apr/21 ]

Also, I don't see in the commit associated with this ticket that the work to actually download the debug symbols at the beginning of each task run has been removed. I'd have expected to see the fetch_debugsymbols_archive step removed from do setup, but I still see it there: https://github.com/mongodb/mongo/blob/0edbd2fe516f44e427f6907f308158d5708de65f/etc/evergreen.yml#L1131

Comment by Andrew Morrow (Inactive) [ 22/Apr/21 ]

robert.guo and siran.wang - Now that this is done, is there a follow-up ticket to make things that currently wait on archive_dist_test_debug wait on archive_dist_test instead?

Comment by Githook User [ 21/Apr/21 ]

Author:

{'name': 'Siran Wang', 'email': 'siran.wang@mongodb.com', 'username': 'vincivans'}

Message: SERVER-54086 support download debug symbol from patch build
Branch: master
https://github.com/mongodb/mongo/commit/0edbd2fe516f44e427f6907f308158d5708de65f

Comment by Brooke Miller [ 16/Feb/21 ]

The original first task included (move the dist-test-debug target to a separate task that's part of the compile_TG but done after compile. The JS tasks depend on compile only, not remaining tasks) was completed by Drew in SERVER-54297.

Generated at Thu Feb 08 05:32:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.