[Perf] Use the perfcomp tool on pull requests

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: Performance Benchmarking
    • None
    • Python Drivers
    • Not Needed
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      DRIVERS-3254 added a perfcomp tool to drivers-evergreen-tools. We should consider enabling it on either PR commits or after the PR, to identify regressions earlier. The Readme has setup and usage instructions. We have an existing PyMongo triage context.

      Example run on a patch that had an intentional slowdown:

      $ ./bin/perfcomp compare --perf-context "PyMongo triage context" --project mongo-python-driver --task perf-8.0-standalone --variant performance-benchmarks 689df49066d4460007dc4961
2025/08/18 10:29:41 Successfully connected to MongoDB Analytics node.
2025/08/18 10:29:41 Successfully retrieved 34 docs from version 689df49066d4460007dc4961.
2025/08/18 10:29:44 🧪 Performance Results
2025/08/18 10:29:44 The following benchmark tests for version 689df49066d4460007dc4961 had statistically significant changes (i.e., |z-score| > 1.96):

| Benchmark                      | Measurement       | % Change | Patch Value | Stable Region                                   | H-Score | Z-Score  |
| ---------                      | -----------       | -------- | ----------- | -------------                                   | ------- | -------  |
| DeepDecoding                   | megabytes_per_sec | -93.0001 | 7.1324      | Avg: 101.8930, Med: 101.6913, Stdev: 0.9885     | 0.9952  | -95.8661 |
...

      Definition of done

      Add perfcomp comments to pull requests.

      Pitfalls

      We should consider the impact of running the performance tests and consider using only one task.

            Assignee:
            Unassigned
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: