Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-4454

Introduce checking benchmark results as part of the release process

    • Type: Icon: Task Task
    • Resolution: Duplicate
    • Priority: Icon: Unknown Unknown
    • None
    • Affects Version/s: None
    • Component/s: Performance
    • Labels:
      None
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      We have some cross-language driver benchmarks run by Evergreen, but there is no automatic detection of regressions, and our release process does not include analyzing manually whether the results are fine or not.

      Checking the results of benchmarks before releasing a new version, and making a decision whether it is acceptable to release based on the results may help us to avoid regressions like JAVA-4452. When done, adding more benchmarks may further increase our chances of spotting a regression before releasing a version.

      The relevant Evergreen docs:


      1 It does not seem like the existing driver benchmarks show the JAVA-4452 regression. There is no data before May 18, but based on the Jira tickets that caused JAVA-4452, the commits to master happened on Apr 14, May 1, Jun 8, Sep 27, and I don't see charts having a stair step shape near those dates.

            Assignee:
            Unassigned Unassigned
            Reporter:
            valentin.kovalenko@mongodb.com Valentin Kavalenka
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: