[SERVER-77371] Parsley displays incorrect patch and task information for resmoke test logs Created: 12/May/23  Updated: 22/May/23  Resolved: 22/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Max Hirschhorn Assignee: [DO NOT ASSIGN] Backlog - Server Development Platform Team (SDP) (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File Screen Recording 2023-05-12 at 7.01.35 PM.mov     PNG File Screen Shot 2023-05-12 at 7.06.57 PM.png    
Issue Links:
Duplicate
duplicates SERVER-77372 resmoke reuses Logkeeper build IDs fo... Closed
Assigned Teams:
Server Development Platform
Participants:

 Description   

For example, the logs for https://parsley.mongodb.com/resmoke/1987b467249673419df9d0d0709b93ea/test/175e7d709facffd8fa1c0902c3073a22?bookmarks=0,6460&shareLine=0 from this Evergreen task failure instead show the patch and task information from a totally unrelated Evergreen project and user. The link itself appears to be correct for navigating from the resmoke test logs back to the Evergreen task page and so only the display text and task status is incorrect.



 Comments   
Comment by Annie Black [ 22/May/23 ]

jk raced Max  

Comment by Annie Black [ 22/May/23 ]

With Max's comment, I'm going to reopen this ticket to move over to Server. Thanks all!

Comment by Max Hirschhorn [ 22/May/23 ]

Thanks all for the replies! The ?metadata=true url parameter is cool and summarizes the inconsistency well.

The buildnum for a particular builder is incremented by the keyval.inc Evergreen command. The intention here is to ensure each (builder, buildnum) combination is only ever used once.

"get buildnumber": &get_buildnumber
  command: keyval.inc
  params:
    key: "${build_variant}_v7.0"
    destination: "builder_num"

It looks like our branching process updates the branch suffix for the key to restart the counter and avoid contention on the document across branches (see also how keyval.inc is implemented). However the builder in the actual credentials file doesn't include the branch name. This means resmoke is providing Logkeeper with (builder, buildnum) combinations which have already been used before. The fix is going to be for the SDP team to edit evergreen/functions/credentials_setup.sh in the mongodb/mongo repository across all of the active branches. CC alexander.neben@mongodb.com

cat > mci.buildlogger << END_OF_CREDS
slavename='${slave}'
passwd='${passwd}'
builder='MCI_${build_variant}'
build_num=${builder_num}
build_phase='${task_name}_${execution}'
END_OF_CREDS

Comment by Julian Edwards [ 22/May/23 ]

Hi max.hirschhorn@mongodb.com I am taking a look at this. Is it possible that the same "builder" was passed to logkeeper for these two tasks? We generate the build ID using the hash of the "builder" and "buildnum" passed in by the requester—otherwise this could be a hash collision.

Comment by Mohamed Khelif [ 22/May/23 ]

max.hirschhorn@mongodb.com The task id is specified by resmoke and we simply query logkeeper with the build id for any given log to get the task we should render here.

Taking a look at the build metadata for both the build and the test it looks like different task ids are being provided.
http://logkeeper.mongodb.org/build/1987b467249673419df9d0d0709b93ea/all?metadata=true
http://logkeeper.mongodb.org/build/1987b467249673419df9d0d0709b93ea/test/175e7d709facffd8fa1c0902c3073a22?metadata=true

Is it possible two different tasks from different projects are writing to the same build id?

Comment by Max Hirschhorn [ 22/May/23 ]

annie.black@mongodb.com, given how the link back to the Evergreen task page goes to the expected location, I don't see how this could be a bug in resmoke. My mental model has it that resmoke must have provided the correct Evergreen task ID to Logkeeper for that link to work correctly. I imagine there's some case where resolving the task display name and status for an Evergreen task ID produces the wrong output.

I haven't seen this come up for other resmoke logs yet I can't say I've specifically been looking for it. I noticed this case because the status says 'Succeeded' when I knew I had clicked on the failing test logs.

Comment by Annie Black [ 22/May/23 ]

max.hirschhorn@mongodb.com have you seen this with other resmoke logs, or was this an isolated incident? If it's isolated then it's likely that resmoke gave us the wrong thing in which case we'll hand the investigation over to them, otherwise something may be broken with what we're asking for. cc alexander.neben@mongodb.com 

Generated at Thu Feb 08 06:35:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.