Uploaded image for project: 'Motor'
  1. Motor
  2. MOTOR-185

When using Change Streams, motor does not shutdown cleanly

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Minor - P4 Minor - P4
    • 1.2.1
    • Affects Version/s: 1.2
    • Component/s: asyncio
    • Labels:
      None
    • Environment:
      Python 3.6.4, Motor 1.2 and pymongo 3.6.0 on macOS Sierra.

      Detailed steps to reproduce the problem, including your code and a full traceback

      I wrote a program to monitor changes to a MongoDB collection using its new Change Stream feature, on Python 3. Here's the MCVE:

      from asyncio import get_event_loop, CancelledError
      from contextlib import suppress
      from motor.motor_asyncio import AsyncIOMotorClient
      
      async def watch(collection):
          async with collection.watch([]) as stream:
              async for change in stream:
                  print(change)
      
      async def cleanup():
          task.cancel()
      
          with suppress(CancelledError):
              await task
      
      if __name__ == '__main__':
          conn = AsyncIOMotorClient()
          loop = get_event_loop()
          task = loop.create_task(watch(conn.database.collection))  # Replace with a real collection.
      
          try:
              loop.run_forever()
      
          except KeyboardInterrupt:
              pass
      
          finally:
              loop.run_until_complete(cleanup())
              loop.shutdown_asyncgens()
              loop.close()

      If you kill that program with CRTL+C, it raises three different exceptions.

      ^Cexception calling callback for <Future at 0x102efea58 state=finished raised InvalidStateError>
      Traceback (most recent call last):
        File "/Users/viotti/motor/lib/python3.6/site-packages/motor/core.py", line 1259, in _next
          change = self.delegate.next()
        File "/Users/viotti/motor/lib/python3.6/site-packages/pymongo/change_stream.py", line 79, in next
          change = self._cursor.next()
        File "/Users/viotti/motor/lib/python3.6/site-packages/pymongo/command_cursor.py", line 292, in next
          raise StopIteration
      StopIteration
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
          result = self.fn(*self.args, **self.kwargs)
        File "/Users/viotti/motor/lib/python3.6/site-packages/motor/core.py", line 1264, in _next
          future.set_exception(StopAsyncIteration())
      asyncio.base_futures.InvalidStateError: invalid state
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/_base.py", line 324, in _invoke_callbacks
          callback(self)
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/futures.py", line 414, in _call_set_state
          dest_loop.call_soon_threadsafe(_set_state, destination, source)
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 620, in call_soon_threadsafe
          self._check_closed()
        File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 357, in _check_closed
          raise RuntimeError('Event loop is closed')
      RuntimeError: Event loop is closed

      I've opened a question on Stackoverflow and it seems those problems trace back to Motor.

      What you expected to happen, and what actually happened.

      A clean shutdown without error messages on the console.

      The exact python version used, with patch level.

      $ python -c "import sys; print(sys.version)"
      3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
      [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

      The exact version of PyMongo used.

      $ python -c "import pymongo; print(pymongo.version); print(pymongo.has_c())"
      3.6.0
      True

      The operating system and version.

      macOs Sierra 10.12.6

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            viotti Rafael Viotti
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: