[Build Failure] Benchmarks are flaky

XMLWordPrintableJSON

    • Type: Build Failure
    • Resolution: Unresolved
    • Priority: Unknown
    • pymongoarrow-next
    • Affects Version/s: None
    • Component/s: pymongoarrow
    • None
    • None
    • Python Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      In https://github.com/mongodb-labs/mongo-arrow/pull/254, two of the benchmarks were slower, even though nothing in the PR should have affected benchmarks. Re-running the benchmark had it pass.

      https://github.com/mongodb-labs/mongo-arrow/actions/runs/12381289983/job/34559542705?pr=254

      Benchmarks that have got worse:
      
      | Change   | Before [7c4cf76b] <bm/merge-target>   | After [92532394] <bm/pr>   |   Ratio | Benchmark (Parameter)                                                                                                                                                                 |
      |----------|---------------------------------------|----------------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
      | +        | 254±10ns                              | 426±60ns                   |    1.68 | benchmarks.ProfileReadArray.time_to_numpy [fv-az740-694/existing-py_home_runner_work_mongo-arrow_mongo-arrow_bindings_python_.tox_benchmark_bin_python-N_DOCS1000]                    |
      | +        | 232±7ns                               | 458±10ns                   |    1.97 | benchmarks.ProfileReadExtensionLarge.time_conventional_arrow [fv-az740-694/existing-py_home_runner_work_mongo-arrow_mongo-arrow_bindings_python_.tox_benchmark_bin_python-N_DOCS1000] |
      

              Assignee:
              Unassigned
              Reporter:
              Steve Silvester
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: