Python 3.10+ may not allow daemon threads in subinterpreters

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Looking at what’s coming in Python 3.9:

      In a subinterpreter, spawning a daemon thread now raises an exception. Daemon threads were never supported in subinterpreters. Previously, the subinterpreter finalization crashed with a Python fatal error if a daemon thread was still running.

      That’s going to make MongoClient incompatible with subinterpreters because we start daemon threads in PeriodicExecutor.

      Proposed solutions:

      1. Make PeriodicExecutor use non-daemon threads but only when running in a subinterpreter.
        • Pros: Doesn't change the behavior of the normal main interpreter case.
        • Cons: Different behavior in main vs sub interpreter is more complex to document and explain. It may not be possible for PeriodicExecutor to check if it's running in the main or sub interpreter.
      2. Make PeriodicExecutor always use non-daemon threads.
        • Pros: Same behavior in main vs sub interpreters.
        • Cons: May further delay main interpreter exit and worsen the problem in PYTHON-1896.

      Note that both proposals will fix the "Py_EndInterpreter: not the last thread" problem described in PYTHON-1634. As part of this change we should test that all threads exits promptly at both main and sub interpreter shutdown.

              Assignee:
              Unassigned
              Reporter:
              Shane Harvey
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: