Context

When running a single-threaded benchmark ( bench-command.py ), PyMongo master is 15-20% slower when compared to PyMongo 3.6.1:

$ python bench-command.py 1
Python: 3.9.13, PyMongo: 4.8.0.dev0, MongoDB: 5.0.15, PID: 99424
Running 10000 operations across 1 thread(s)
command: 1.33s
find_one: 1.95s
insert_one: 1.88s

vs 3.6.1:

$ python bench-command.py 1  
Python: 3.9.13, PyMongo: 3.6.1, MongoDB: 5.0.15, PID: 98867
Running 10000 operations across 1 thread(s)
command: 1.11s
find_one: 1.66s
insert_one: 1.66s

When running a 100-thread benchmark, PyMongo master is 40-50% slower when compared to PyMongo 3.6.1:

$ python bench-command.py 100
Python: 3.9.13, PyMongo: 4.8.0.dev0, MongoDB: 5.0.15
Running 10000 operations across 100 thread(s)
command: 1.65s
find_one: 1.76s
insert_one: 1.53s

vs 3.6.1:

$ python bench-command.py 100
Python: 3.9.13, PyMongo: 3.6.1, MongoDB: 5.0.15
Running 10000 operations across 100 thread(s)
command: 1.12s
find_one: 1.20s
insert_one: 1.10s

Here's another representation of the benchmark results, this time with a few more PyMongo versions:
1 thread:

100 threads:

Note these tests were run locally on macos where master was commit d6bf0e1e78c3287149ef56d9718bb6098aa6e41f (PYTHON-4264).

Definition of done

Investigate the performance regressions and bottlenecks in the the driver and identify potential optimizations.

We should be able to improve both single-threaded and multi-threaded performance. One goal is that the 100-thread benchmark should complete faster than the single threaded version just as it does in PyMongo 3.5.1.

Pitfalls

It could be impossible to match 3.5.1 performance because many features have been added since then which we can't remove (sessions, retryable reads/writes, CSOT, maxConnecting, logging, OP_MSG, etc...) but we should be able to get closer the performance of 3.7 (or ideally 3.5).