Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1428

Pool may exceed maxPoolSize sockets when minPoolSize==maxPoolSize

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Minor - P4 Minor - P4
    • 3.6.1
    • Affects Version/s: None
    • Component/s: Internal
    • None

      Failed on travis: https://travis-ci.org/mongodb/mongo-python-driver/jobs/309136359#L1716

      FAIL: test_max_idle_time_reaper (test_client.TestClient)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/travis/build/mongodb/mongo-python-driver/test/test_client.py", line 293, in test_max_idle_time_reaper
          self.assertEqual(1, len(server._pool.sockets))
      AssertionError: 1 != 2
      

      There is a race in the Pool.remove_stale_sockets method. We use this code to ensure there are at least minPoolSize sockets in the Pool:

              while len(
                      self.sockets) + self.active_sockets < self.opts.min_pool_size:
                  sock_info = self.connect()
                  with self.lock:
                      self.sockets.add(sock_info)
      

      However, if another thread is in the process of checking out a socket at the same time we may add more that minPoolSize sockets. Imagine minPoolSize=1 and maxPoolSize=1:
      1) Topology calls remove_stale_sockets. Pool size = 0
      2) No sockets are in the pool. remove_stale_sockets creates a connection but has not added it to the Pool. Pool size is still 0.
      3) Another thread calls pool.get_socket({}) and creates a socket. Pool size = 1
      4) remove_stale_sockets resumes execution adds the new socket to the pool. Pool size = 2

      remove_stale_sockets must acquire the Pool's socket semaphore before creating a socket to ensure that maxPoolSize is respected.

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: