[SERVER-81306] core-analyzer bug fixes Created: 21/Sep/23 Updated: 03/Nov/23 Resolved: 27/Sep/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Trevor Guidry | Assignee: | Trevor Guidry |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Participants: | |||||
| Linked BF Score: | 61 | ||||
| Description |
|
This fixes a couple of bugs in the core analyzer currently.
Old description below: Currently the task is generated if there are any core dumps found on the task. Sometimes we upload core dumps from processes that are not mongo binaries. This can lead to no analysis being done if the only core dumps there are from non-mongo processes. We need to be smarter about when we generate the tasks and check if at least one of the core dumps is from a known binary
An example failure caused by this issue is here https://spruce.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_all_feature_flags_display_replica_sets_abe6f7a64d785277fb223958957252c6f8f89027_23_09_21_11_09_00 |
| Comments |
| Comment by Githook User [ 27/Sep/23 ] | |
|
Author: {'name': 'Trevor Guidry', 'email': 'trevor.guidry@mongodb.com', 'username': ''}Message: | |
| Comment by Trevor Guidry [ 21/Sep/23 ] | |
|
max.hirschhorn@mongodb.com Thanks for commenting, I was going to naively use the file name. I will have to think about this more now. | |
| Comment by Max Hirschhorn [ 21/Sep/23 ] | |
|
trevor.guidry@mongodb.com, would you please clarify how you intend to detect whether a core dump was generated from a known MongoDB binary? An approach based on the filename won't be possible. This is because %e in kernel.core_pattern=dump_%e.%p.core is substituted with the thread name when the process crashes rather than the process name. In the linked Evergreen task, BackgroundSync refers to the name of a thread related to the replication subsystem in mongod. It is a core dump which must successfully analyzed.
The core dump contains the contents of /proc/<pid>/exe and this information is (i) displayed in a "Core was generated by" message by gdb and (ii) accessible programmatically within gdb from the info proc exe command. You may be interested in looking at this logic in failed_unittests_gather.sh as an approach for doing (i). |