Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1269

Explicitly closed cursors may not be killed before termination

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 3.5
    • None
    • None
    • Minor Change

    Description

      As a result of PYTHON-799, cursors are added to a queue upon closure and killCursors on their IDs are issued periodically in the background. In cases where an uniterated cursor is queued for closure shortly before termination (for instance, in a signal handler), the process can exit before the next execution of the periodic cleanup thread, leaving open cursors on the server. This can have negative consequences in e.g. a sharded cluster, where open cursors can block chunk migrations. The latter problem is worsened in cases where customers have increased the server-side cursorTimeoutMillis for operational reasons.

      A minimal reproduction script is attached, which quite reliably leaves a cursor open on the server after termination as shown below. Tests were conducted with pymongo 3.4.0 on a 3.4.3 replica set and sharded cluster.

      > db.serverStatus().metrics.cursor
      {
      	"timedOut" : NumberLong(0),
      	"open" : {
      		"noTimeout" : NumberLong(0),
      		"pinned" : NumberLong(0),
      		"total" : NumberLong(1)
      	}
      }
      

      Sleeping for a few seconds after closing the cursor avoids this issue, as does adding one final execution of the cleanup function after the join in the periodic executor's exit handler. Registering a custom CursorManager to close the cursors synchronously also prevents orphaned cursors, although I believe that this approach would again be susceptible to PYTHON-799.

      Thanks!
      Bernard

      Attachments

        Issue Links

          Activity

            People

              shane.harvey@mongodb.com Shane Harvey
              bernard.gorman@mongodb.com Bernard Gorman
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: