Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1626

Server closes live cursors

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6, 3.6.1, 3.7.1
    • Component/s: None
    • None

      Hi,
      we have some code* that accepts a list of MongoDB servers (3.6.x), databases, and collections and proceeds to download these. We did this a while ago and it worked just fine. When we tried this a few weeks ago, downloading larger collections often ended with CursorNotFound during iteration. We did not time out, it could happen as soon as a few seconds in. We tried making batch sizes smaller, to no avail (same for no_cursor_timeout and a few other timeout settings).

      At first we thought we faced this server issue SERVER-34810, but updating all our MongoDBs to 3.6.6 didn't resolved this. What worked was downgrading PyMongo to 3.5.1. We then proceeded with a git bisect, which revealed this commit to be the offending one.

      https://github.com/mongodb/mongo-python-driver/commit/c1ec855060c0d79613f10e984086c6d400ccc312

      Sadly, it's quite a large commit, so I'm not sure where to start.

      We were not able to replicate this in a dummy environment. My guess is that since we're accessing ~15 collections in ~4 different mongo servers, the driver closes a cursor it believes was an old one (for a given server), but it's actually a current one. If you could help me instrument this, I could give you more diagnostics.

      • the code pretty much just creates MongoClient, does a `.find()` and iterates over this cursor and serialises each document into a file

        1. bug_jira_seq.py
          1.0 kB
        2. traceback.txt
          0.8 kB
        3. exc_details.txt
          1.0 kB

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            ondrejk Ondrej Kokes
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: