Uploaded image for project: 'Motor'
  1. Motor
  2. MOTOR-58

max_pool_size not enforced

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 0.4
    • Affects Version/s: 0.2
    • Component/s: None
    • Labels:
      None

      If many greenlets call MotorPool.get_socket at once, the pool can expand beyond its max_pool_size (100 by default). The bug was introduced in Motor 0.2 when I enabled async DNS resolution:

      https://github.com/mongodb/motor/commit/d2d811fb

      create_connection() increments its counter before attempting to connect. It had been in a critical section until then. Unfortunately, once resolve() became a yielding operation, the greenlet could be interrupted when it called resolve() and the one critical section was split into two.

      One after another, greenlets see that there is room in the pool, then they reach the resolve() call and yield. Once they all resume after the yield there are more greenlets than max_pool_size but they don't check again if there is room.

      The impact is low because extra sockets above max_pool_size are closed as soon as they're returned to the pool, so the pool does not remain over-sized forever. This is why tests like test_max_size passed even after the bug was introduced.

      The solution is to increment the pool counter before the first yield at resolve().

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: