-
Type:
Task
-
Resolution: Won't Fix
-
Priority:
Unknown
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
sophiayangDB has created PR #2804: PYTHON-5671: Use fine-grained locks in the connection pool to reduce contention in mongo-python-driver
Issue Text:
PYTHON-5671
-
- Changes in this PR
Based on investigation inPYTHON-5007, using fine-grained locks improved thread efficiency by about 20% in a small read scenario. This PR seeks to implement the suggestions.
- Changes in this PR
Previously, code in `pool.py` was protected by the sole lock `self.lock` and its two conditionals. Now, there are six individual locks that are the sole protectors of the following:
- `self._conns_lock`
Protects `self.conns` - `self._active_contexts_lock`
Protects `self.active_contexts` as well as the subsequent updates to `self.__pinned_sockets`, `self.active_sockets`, `next_connection_id`, `self.ncursors`, and `self.ntxns` - `self.size_cond`
Protects `self.requests` and `self.gen` - `self._max_connecting_cond`
Protects `self._pending` - `self._operation_count_lock`
Protects `self.operation_count` - `self.lock`
Protects `self.state`
Other strategies used to mitigate lock contention include:
- Moving lines that do not require a lock to be performed before lock acquisition
- Use of temporary, local variables to avoid nested lock contention
- Removing lock acquisition from `update_is_writable()` as its sole caller already has the lock
- Copying `self.active_contexts` to update its references when additional additions/deletions to the set should not be operated upon or their absence does not modify behavior, avoids errors that arise if calling on the original and it's length/size changes
-
- Test Plan
Tests already exist in `main/test/test_pooling.py` and `test/asynchronous/test_pooling.py`. Since this was only a code refactor, no additional tests were added.
- Test Plan
-
- Checklist
<!-- Do not delete the items provided on this checklist. -->
- Checklist
-
-
- Checklist for Author
-
- [ ] Did you update the changelog (if necessary)?
- [yes] Is there test coverage?
- [n/a] Is any followup work tracked in a JIRA ticket? If so, add link(s).
-
-
- Checklist for Reviewer
-
- [ ] Does the title of the PR reference a JIRA Ticket?
- [ ] Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
- [ ] Is all relevant documentation (README or docstring) updated?