Uploaded image for project: 'Motor'
  1. Motor
  2. MOTOR-66

Motor Deadlocks when Running Parallel Requests

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 0.4
    • Fix Version/s: 0.4.1
    • Component/s: None
    • Labels:
      None
    • Environment:
      Ubuntu 14.04 / 15.04
      Python 2.7
      Tornado 4.1
      Motor 0.4
      PyMongo 2.8
    • # Replies:
      7
    • Last comment by Customer:
      true

      Description

      I've come across an issue where Motor will deadlock under heavy load or when running parallel queries.

      If I put my query in a module-level function like:

      @gen.coroutine
      def get_data():
          cur1 = _client.test.testcol1.find()
          cur2 = _client.test.testcol2.find()
              
          data1, data2 = yield [cur1.to_list(length=None), cur2.to_list(length=None)]
          
          raise gen.Return((data1, data2))
      

      And then call that function from a RequestHandler like:

      class MainHandler(web.RequestHandler):
          @gen.coroutine
          def get(self):
              yield get_data()
              
              self.write("done")
      

      Motor will always deadlock at:

      # ThreadID: 140520054396672
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/motor/__init__.py", line 683, in call_method
        result = sync_method(self.delegate, *args, **kwargs)
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/cursor.py", line 1020, in _refresh
        self.__uuid_subtype))
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/cursor.py", line 933, in __send_message
        res = client._send_message_with_response(message, **kwargs)
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1597, in _send_message_with_response
        self._ensure_connected()
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1318, in _ensure_connected
        self.__ensure_monitor()
      File: "/home/bmiller/opt/python/local/lib/python2.7/site-packages/pymongo/mongo_replica_set_client.py", line 1087, in __ensure_monitor
        self.__monitor_lock.acquire()
      

      However, if I put the code directly inside the get method of my RequestHandler it will work fine.

      Here's a simple Tornado app that I threw together that will reproduce the issue:
      https://gist.github.com/brentalanmiller/efb184b495cdb92d4524

      Thanks,
      Brent

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Days since reply:
                  3 years, 31 weeks, 4 days ago
                  Date of 1st Reply: