Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-4454

Introduce checking benchmark results as part of the release process

    XMLWordPrintable

Details

    • Task
    • Status: Backlog
    • Unknown
    • Resolution: Unresolved
    • None
    • None
    • Performance
    • None

    Description

      We have some cross-language driver benchmarks run by Evergreen, but there is no automatic detection of regressions, and our release process does not include analyzing manually whether the results are fine or not.

      Checking the results of benchmarks before releasing a new version, and making a decision whether it is acceptable to release based on the results may help us to avoid regressions like JAVA-4452. When done, adding more benchmarks may further increase our chances of spotting a regression before releasing a version.

      The relevant Evergreen docs:


      1 It does not seem like the existing driver benchmarks show the JAVA-4452 regression. There is no data before May 18, but based on the Jira tickets that caused JAVA-4452, the commits to master happened on Apr 14, May 1, Jun 8, Sep 27, and I don't see charts having a stair step shape near those dates.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              valentin.kovalenko@mongodb.com Valentin Kavalenka
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: