Uploaded image for project: 'Motor'
  1. Motor
  2. MOTOR-66

Motor Deadlocks when Running Parallel Requests

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 0.4.1
    • Affects Version/s: 0.4
    • Component/s: None
    • None
    • Environment:
      Ubuntu 14.04 / 15.04
      Python 2.7
      Tornado 4.1
      Motor 0.4
      PyMongo 2.8
    • None
    • None
    • None
    • None
    • None
    • None

      I've come across an issue where Motor will deadlock under heavy load or when running parallel queries.

      If I put my query in a module-level function like:

      @gen.coroutine
def get_data():
    cur1 = _client.test.testcol1.find()
    cur2 = _client.test.testcol2.find()
        
    data1, data2 = yield [cur1.to_list(length=None), cur2.to_list(length=None)]
    
    raise gen.Return((data1, data2))

      And then call that function from a RequestHandler like:

      class MainHandler(web.RequestHandler):
    @gen.coroutine
    def get(self):
        yield get_data()
        
        self.write("done")

      Motor will always deadlock at:

      # ThreadID: 140520054396672
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/motor/__init__.py", line 683, in call_method
  result = sync_method(self.delegate, *args, **kwargs)
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/cursor.py", line 1020, in _refresh
  self.__uuid_subtype))
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/cursor.py", line 933, in __send_message
  res = client._send_message_with_response(message, **kwargs)
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1597, in _send_message_with_response
  self._ensure_connected()
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1318, in _ensure_connected
  self.__ensure_monitor()
File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1087, in __ensure_monitor
  self.__monitor_lock.acquire()

      However, if I put the code directly inside the get method of my RequestHandler it will work fine.

      Here's a simple Tornado app that I threw together that will reproduce the issue:
      https://gist.github.com/brentalanmiller/efb184b495cdb92d4524

      Thanks,
      Brent

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            bmiller@bloomenergy.com Brent Miller
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: