Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-4292

Investigate apparent minor perf regression in SSL benchmarks

    • Type: Icon: Task Task
    • Resolution: Cannot Reproduce
    • Priority: Icon: Unknown Unknown
    • None
    • Affects Version/s: None
    • Component/s: Performance
    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      Context

      Investigate apparent minor perf regression in SSL benchmarks due to PYTHON-3175:

      I suspect the issue is caused by the extra overhead of calling wait_for_read().

      In the PR for PYTHON-3175 I noted this change https://github.com/mongodb/mongo-python-driver/pull/1465#issuecomment-1896673108:

      Looking at the perf results it does appear this results in a ~5% decrease. Locally with 100 threads I see a decrease of somewhere between 5-10%:

      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.61s user 8.06s system 129% cpu 11.320 total
      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.65s user 8.13s system 130% cpu 11.297 total
      $ git:(master) time python bench-100-threads.py 
      python bench-100-threads.py  6.86s user 8.52s system 129% cpu 11.842 total
      

      vs

      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.18s user 8.52s system 127% cpu 12.323 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.26s user 8.88s system 130% cpu 12.320 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-100-threads.py 
      python bench-100-threads.py  7.34s user 9.02s system 132% cpu 12.335 total
      

      With a single threaded benchmark the total execution time is about the same but the user and system CPU time goes up by 5%:

      $ git:(master) time python bench-single-thread.py
      1.80s user 0.34s system 50% cpu 4.259 total
      $ git:(master) time python bench-single-thread.py
      1.81s user 0.34s system 50% cpu 4.267 total
      $ git:(master) time python bench-single-thread.py
      1.85s user 0.36s system 51% cpu 4.277 total
      

      VS

      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.99s user 0.41s system 56% cpu 4.277 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.97s user 0.42s system 56% cpu 4.239 total
      $ git:(NoahStapp-PYTHON-3175) time python bench-single-thread.py
      1.97s user 0.41s system 55% cpu 4.254 total
      

      Here's the single-threaded benchmark:

      from pymongo import MongoClient
      client = MongoClient()
      N_ITERATIONS = 50000
      
      def main():
          for _ in range(N_ITERATIONS):
              client.admin.command('ping')
      
      if __name__ == "__main__":
          main()
      

      Definition of done

      Remove regressions on all benchmarks when compared to 4.6.2.

      Pitfalls

      Note that some benchmarks have high run to run variance which makes it a little harder to spot true regressions.

        1. bench-100-threads.py
          0.4 kB
          Shane Harvey
        2. Screenshot 2024-03-21 at 3.30.33 PM.png
          375 kB
          Shane Harvey

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: