[Perf] Use the perfcomp tool on pull requests

XMLWordPrintableJSON

    • None
    • Python Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      DRIVERS-3254 added a perfcomp tool to drivers-evergreen-tools. We should consider enabling it on either PR commits or after the PR, to identify regressions earlier. The Readme has setup and usage instructions.

      Example run on a patch that had an intentional slowdown:

      $ ./bin/perfcomp compare --perf-context "PyMongo triage context" --project mongo-python-driver --task perf-8.0-standalone --variant performance-benchmarks 689df49066d4460007dc4961
2025/08/18 10:29:41 Successfully connected to MongoDB Analytics node.
2025/08/18 10:29:41 Successfully retrieved 34 docs from version 689df49066d4460007dc4961.
2025/08/18 10:29:44 🧪 Performance Results
2025/08/18 10:29:44 The following benchmark tests for version 689df49066d4460007dc4961 had statistically significant changes (i.e., |z-score| > 1.96):

| Benchmark                      | Measurement       | % Change | Patch Value | Stable Region                                   | H-Score | Z-Score  |
| ---------                      | -----------       | -------- | ----------- | -------------                                   | ------- | -------  |
| DeepDecoding                   | megabytes_per_sec | -93.0001 | 7.1324      | Avg: 101.8930, Med: 101.6913, Stdev: 0.9885     | 0.9952  | -95.8661 |
...

      Definition of done

      Add perfcomp comments to pull requests.

      Pitfalls

      We should consider the impact of running the performance tests and consider using only one task.

            Assignee:
            Unassigned
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: