POC: automated flaky test tracking

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: Tooling
    • None
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Use Case

      As discussed in the Sep 12, 2025 retro, we would like to expore automated flaky test tracking so that we can enable Mocha's automated test retry capabilities to make CI greener.

      User Experience

      • n/a

      Dependencies

      • n/a

      Risks/Unknowns

      • n/a

      Acceptance Criteria

      Implementation Requirements

      We'd like a tool that has the following capabilities:

      • Automatically retries a test, if the test fails.  If the test passes on a second run, automatically marks the test as flaky.
        • Captures the stacktrace and the error, as well as test logs (ideally)
        • Records test metatdata (environment + configuration)
        • Posts the flaky test somewhere (Free tier cluster? Integration with Jira?)
      • Probably only enabled in CI
      • Has a mechanism for viewing flaky tests so we can monitor for flaky test counts

      Testing Requirements

      • n/a

      Documentation Requirements

      • n/a

      Follow Up Requirements

      • n/a

            Assignee:
            Unassigned
            Reporter:
            Bailey Pearson
            None
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: