ExportXMLWordPrintableJSON

    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      The performance suite measures data throughput: BSON encoding and decoding, bulk
      CRUD, and GridFS. Nothing measures connection checkout, so changes to pool or
      topology locking are invisible to CI.

      PYTHON-5898 had to measure both of its proposed changes with throwaway
      benchmarks. The pool change improved checkout throughput 6.6% at concurrency 20;
      the proposed topology reader/writer lock made server selection ~30% slower and
      was abandoned. Neither result would have been visible in CI.

      Definition of done

      • A benchmark measuring connection checkout throughput under concurrency,
        following the existing PerformanceTest pattern in
        test/performance/perf_test.py, with an async counterpart in
        async_perf_test.py.
      • Reports operations_per_sec in the metrics list rather than reusing
        megabytes_per_sec, which does not apply.
      • Covers both the uncontended path and the wait-queue path (concurrency above
        maxPoolSize).
      • Runs in the existing performance-benchmarks variant, no new hardware, no
        meaningful added wall-clock time.
      • Results appear in Evergreen's performance tooling and are comparable across
        commits.

      Pitfalls

      • Driving checkout through ordinary operations measures the network and server,
        not the driver. In PYTHON-5898, checkout ran ~190k ops/s in isolation while
        real operations against localhost sustained ~8k ops/s. Exercise checkout
        directly.
      • Do not instrument locks to report contention percentages. PYTHON-5898
        measured topology lock contention at 0.016% of worker time at concurrency 20,
        falling to 0.003% at 100 – not an actionable signal, and it requires patching
        internals that later break.
      • Size the case so run-to-run variance is well below the effect sizes worth
        catching. The two changes above were 6.6% and 30%.
      • Async and sync differ structurally: a single-threaded event loop versus threads
        under the GIL. The same workload can behave differently; cover both.
      • Fix concurrency level and iteration count rather than deriving them from the
        host, or results cannot be compared against history.
      • Anything comparing two builds must pin the interpreter version and confirm the
        C extensions are present on both. An early PYTHON-5898 comparison was invalid
        because the two sides ran different Python versions.

            Assignee:
            Unassigned
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: