Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-4292

Perf regression when reading large SSL responses from multiple threads

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 4.11
    • Affects Version/s: None
    • Component/s: Performance
    • None
    • Python Drivers
    • Not Needed

      Context

      Investigate apparent minor perf regression in SSL benchmarks due to PYTHON-3175:

      I suspect the issue is caused by the extra overhead of calling wait_for_read().

      In the PR for PYTHON-3175 I noted this change https://github.com/mongodb/mongo-python-driver/pull/1465#issuecomment-1896673108:

      Looking at the perf results it does appear this results in a ~5% decrease. Locally with 100 threads I see a decrease of somewhere between 5-10%:

      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.61s user 8.06s system 129% cpu 11.320 total
      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.65s user 8.13s system 130% cpu 11.297 total
      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.86s user 8.52s system 129% cpu 11.842 total
      

      vs

      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.18s user 8.52s system 127% cpu 12.323 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.26s user 8.88s system 130% cpu 12.320 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.34s user 9.02s system 132% cpu 12.335 total
      

      With a single threaded benchmark the total execution time is about the same but the user and system CPU time goes up by 5%:

      $ git:(master) time python bench-single-thread.py
      1.80s user 0.34s system 50% cpu 4.259 total
      $ git:(master) time python bench-single-thread.py
      1.81s user 0.34s system 50% cpu 4.267 total
      $ git:(master) time python bench-single-thread.py
      1.85s user 0.36s system 51% cpu 4.277 total
      

      VS

      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.99s user 0.41s system 56% cpu 4.277 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.97s user 0.42s system 56% cpu 4.239 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.97s user 0.41s system 55% cpu 4.254 total
      

      Here's the single-threaded benchmark:

      from pymongo import MongoClient
      client = MongoClient()
      N_ITERATIONS = 50000
      
      def main():
          for _ in range(N_ITERATIONS):
              client.admin.command('ping')
      
      if __name__ == "__main__":
          main()
      

      Definition of done

      Remove regressions on all benchmarks when compared to 4.6.2.

      Pitfalls

      Note that some benchmarks have high run to run variance which makes it a little harder to spot true regressions.

        1. bench-100-threads.py
          0.4 kB
        2. bench-100-threads-find-large.py
          0.9 kB
        3. Screenshot 2024-03-21 at 3.30.33 PM.png
          Screenshot 2024-03-21 at 3.30.33 PM.png
          375 kB

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: