Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1482

Critical Fault in max_idle_time_ms checking logic (causes connections to close instantly)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 3.5, 3.5.1, 3.6
    • 3.6.1
    • Config
    • None
    • Python 2.7.14

    Description

      Hi,

      I'm really sorry for the stream of consciousness style reporting. I'm in extreme hurry currently. I think this problem affecting every user with Python 2.x and using maxIdleTimeMS parameter. In our case it was causing half a million of unneccasry reconnections in an hour and exploding our Mongo Atlas log with reconnections (https://jira.mongodb.org/browse/MMSSUPPORT-19979).

      I was investigation why our connection pools we're always closed instantly after query. Even we had set maxIdleTimeMS to 10000 ms and found some problems in Pool.remove_stale_sockets method.

      idle_time_ms = 1000 * _time() - sock_info.last_checkin
      Should be:
      idle_time_ms = 1000 * (_time() - sock_info.last_checkin)
      I can provided more details why If this is not self evident. This same fault logic can be found in the _check method in the Pool class.

      But then there is another problem that is python 2.x spesific.
      pymongo: monotonic.py tries to import:
      from time import monotonic as time

      This will always fail because the file it self is called monotonic (circular import). Other major problem is that monotonic returns time in seconds after epoc and python time as ms from epoc. So actually the "1000 * " should be removed entirely when calculating idle_time_ms.

      I'm sure that one reason why this has happened is that the variable is called "self.opts.max_idle_time_ms" but the value is actually in seconds. (caused by the validate_timeout_or_none validation common)

      All the best,
      Jussi Kauppila
      vainu.io

      Attachments

        Issue Links

          Activity

            People

              shane.harvey@mongodb.com Shane Harvey
              jussi@vainu.io Jussi Kauppila
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: