[JAVA-4454] Introduce checking benchmark results as part of the release process Created: 26/Jan/22  Updated: 28/Sep/23  Resolved: 28/Sep/23

Status: Closed
Project: Java Driver
Component/s: Performance
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Valentin Kavalenka Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates JAVA-5065 Standardize performance testing infra... Closed
Related
is related to JAVA-4462 Introduce a benchmark that could have... Backlog
is related to JAVA-4452 Improve performance characteristics o... Closed
Quarter: FY24Q3
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

We have some cross-language driver benchmarks run by Evergreen, but there is no automatic detection of regressions, and our release process does not include analyzing manually whether the results are fine or not.

Checking the results of benchmarks before releasing a new version, and making a decision whether it is acceptable to release based on the results may help us to avoid regressions like JAVA-4452. When done, adding more benchmarks may further increase our chances of spotting a regression before releasing a version.

The relevant Evergreen docs:


1 It does not seem like the existing driver benchmarks show the JAVA-4452 regression. There is no data before May 18, but based on the Jira tickets that caused JAVA-4452, the commits to master happened on Apr 14, May 1, Jun 8, Sep 27, and I don't see charts having a stair step shape near those dates.


Generated at Thu Feb 08 09:02:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.