Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1174

Clarify usage of PyMongo with multiprocessing.

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • 3.4
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None

      I upgraded to Ubuntu 16.04 and started my iibench client.
      https://github.com/mdcallag/mytools/blob/master/bench/ibench/iibench.py

      I immediately get this warning.
      /usr/lib/python2.7/dist-packages/pymongo/topology.py:74: UserWarning: MongoClient opened before fork. Create MongoClient with connect=False, or create client after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#using-pymongo-with-multiprocessing>
      "MongoClient opened before fork. Create MongoClient "

      Then I read the FAQ including this:
      http://api.mongodb.com/python/current/faq.html#using-pymongo-with-multiprocessing

      My code doesn't call fork, but it uses multiprocessing which calls fork for me. The new rules for PyMongo are extremely limiting. I have to do all my calls to fork and multiprocessing.Process up front. Imagine if I had to do all of my database queries at process start. What if I want to make occasional calls to multiprocessing.Process as my program runs? I can't do that given the new limits.

      Can I get a version of PyMongo that doesn't have so much value added under the covers so I can use multiprocessing without these limits?

            Assignee:
            bernie@mongodb.com Bernie Hackett
            Reporter:
            mdcallag Mark Callaghan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: