mongo-python-driver - PR #2804: PYTHON-5671: Use fine-grained locks in the connection pool to reduce contention

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Won't Fix
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      sophiayangDB has created PR #2804: PYTHON-5671: Use fine-grained locks in the connection pool to reduce contention in mongo-python-driver

      Issue Text:
      PYTHON-5671

        1. Changes in this PR
          Based on investigation in PYTHON-5007, using fine-grained locks improved thread efficiency by about 20% in a small read scenario. This PR seeks to implement the suggestions.

      Previously, code in `pool.py` was protected by the sole lock `self.lock` and its two conditionals. Now, there are six individual locks that are the sole protectors of the following:

      • `self._conns_lock`
        Protects `self.conns`
      • `self._active_contexts_lock`
        Protects `self.active_contexts` as well as the subsequent updates to `self.__pinned_sockets`, `self.active_sockets`, `next_connection_id`, `self.ncursors`, and `self.ntxns`
      • `self.size_cond`
        Protects `self.requests` and `self.gen`
      • `self._max_connecting_cond`
        Protects `self._pending`
      • `self._operation_count_lock`
        Protects `self.operation_count`
      • `self.lock`
        Protects `self.state`

      Other strategies used to mitigate lock contention include:

      • Moving lines that do not require a lock to be performed before lock acquisition
      • Use of temporary, local variables to avoid nested lock contention
      • Removing lock acquisition from `update_is_writable()` as its sole caller already has the lock
      • Copying `self.active_contexts` to update its references when additional additions/deletions to the set should not be operated upon or their absence does not modify behavior, avoids errors that arise if calling on the original and it's length/size changes
        1. Test Plan
          Tests already exist in `main/test/test_pooling.py` and `test/asynchronous/test_pooling.py`. Since this was only a code refactor, no additional tests were added.
        1. Checklist
          <!-- Do not delete the items provided on this checklist. -->
          1. Checklist for Author
      • [ ] Did you update the changelog (if necessary)?
      • [yes] Is there test coverage?
      • [n/a] Is any followup work tracked in a JIRA ticket? If so, add link(s).
          1. Checklist for Reviewer
      • [ ] Does the title of the PR reference a JIRA Ticket?
      • [ ] Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
      • [ ] Is all relevant documentation (README or docstring) updated?

            Assignee:
            Steve Silvester
            Reporter:
            TPM Jira Automations Bot
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: