Benchmark check fails for stacked PRs due to existing patch context in Evergreen

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Issue Summary

      The benchmark check in Evergreen is failing for stacked PRs when an existing patch for the same commit SHA is present. The error message indicates that the status reported is not in the context of the current PR, which can cause confusion and test failures, especially when tests depend on PR-specific values.

      Context

      • A user encountered consistent failures in their PR's Evergreen tasks (example).
      • The Evergreen system detected an existing patch for the commit SHA, leading to the following message:
        There is an existing patch(es) for this commit SHA:
        
            Evergreen patch with PR
        
        Please note that the status that is posted is not in the context of this PR but rather the (latest) existing patch and that may affect some tests that may depend on the particular PR. If your tests do not rely on any PR-specific values (like base or head branch name) then your tests will report the same status. If you would like a patch to run in the context of this PR and abort the other(s), comment 'evergreen retry'.
        
      • The issue is reproducible with stacked diffs/PRs, as confirmed by the user.
      • Reference to a related issue: DEVPROD-26260.
      • It was discussed that resolving DEVPROD-26260 (removing custom git calls in the benchmark check) may address this, as Evergreen would then handle base commit retrieval more robustly.
      • There is a consensus that stacked PRs are common, and the system should either support this use case or provide a clearer error message to guide users.

      Proposed Solution

      • Investigate whether resolving DEVPROD-26260 will eliminate this failure for stacked PRs.
      • If not, update the benchmark check logic to better support stacked PRs, or at minimum, enhance the error message to clarify that the error may resolve itself once underlying PRs are merged.
      • Consider documenting this edge case for users working with stacked PRs.

      Original Slack thread: https://mongodb.slack.com/archives/C08D8RX2K5H/p1769069254372819
      This ticket was generated by AI from a Slack thread.

            Assignee:
            Unassigned
            Reporter:
            Austin Hartschen
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: