Add a concurrent connection checkout benchmark to the performance suite

XMLWordPrintableJSON

    • None
    • Python Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      The performance suite measures data throughput: BSON encoding and decoding, bulk
      CRUD, and GridFS. Nothing measures connection checkout, so changes to pool or
      topology locking are invisible to CI.

      PYTHON-5898 had to measure both of its proposed changes with throwaway
      benchmarks. The pool change improved checkout throughput 6.6% at concurrency 20;
      the proposed topology reader/writer lock made server selection ~30% slower and
      was abandoned. Neither result would have been visible in CI.

      Definition of done

      • A benchmark measuring connection checkout throughput under concurrency,
        following the existing PerformanceTest pattern in
        test/performance/perf_test.py, with an async counterpart in
        async_perf_test.py.
      • Reports operations_per_sec in the metrics list rather than reusing
        megabytes_per_sec, which does not apply.
      • Covers both the uncontended path and the wait-queue path (concurrency above
        maxPoolSize).
      • Runs in the existing performance-benchmarks variant, no new hardware, no
        meaningful added wall-clock time.
      • Results appear in Evergreen's performance tooling and are comparable across
        commits.

      Pitfalls

      • Driving checkout through ordinary operations measures the network and server,
        not the driver. In PYTHON-5898, checkout ran ~190k ops/s in isolation while
        real operations against localhost sustained ~8k ops/s. Exercise checkout
        directly.
      • Do not instrument locks to report contention percentages. PYTHON-5898
        measured topology lock contention at 0.016% of worker time at concurrency 20,
        falling to 0.003% at 100 – not an actionable signal, and it requires patching
        internals that later break.
      • Size the case so run-to-run variance is well below the effect sizes worth
        catching. The two changes above were 6.6% and 30%.
      • Async and sync differ structurally: a single-threaded event loop versus threads
        under the GIL. The same workload can behave differently; cover both.
      • Fix concurrency level and iteration count rather than deriving them from the
        host, or results cannot be compared against history.
      • Anything comparing two builds must pin the interpreter version and confirm the
        C extensions are present on both. An early PYTHON-5898 comparison was invalid
        because the two sides ran different Python versions.

            Assignee:
            Unassigned
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: