Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-434

PyMongo locks up forked application

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.3
    • Component/s: None
    • Labels:
      None

      PyMongo protects against forking by checking the pid periodically, but still acquires a lock on the pool prior to resetting. Unfortunately, this means that if the parent process had the lock in some thread while another thread forks, pymongo tries (correctly) to reset its pool, but will hang indefinitely waiting for the lock that was owned by another thread (but in the new fork is not).

      Attached script demonstrates this issue; passing "solve" as an argument when running the script shows one possible solution (forcibly breaking the lock).

      And now some commentary on the proposed solution... Breaking a lock is dangerous. I am assuming that all code in pymongo checks the pid and calls a reset prior to acquiring the pool's lock. If it does, then this code is safe:

      1. The documentation for threading.Lock says that the thread releasing the lock does not need to be the same as the thread that originally locked it.

      2. The resetLock is protected from the original condition because it is only acquired if we know we are in a fork. There is still an extremely small window of opportunity here, but most applications (in mine, which is a task framework spinning off new tasks) should be covered by this guarantee (that pymongo will only lock up if it was already resetting from a fork when you fork again).

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            shadowman131 Walt Woods
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: