Uploaded image for project: 'Motor'
  1. Motor
  2. MOTOR-135

Fine tuning of ThreadPoolExecutor workers number

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • 1.2
    • Affects Version/s: None
    • Component/s: asyncio, tornado
    • Labels:
      None

      Motivation
      Recently we’ve updated motor from 0.6 to 1.1 and had memory issues on production. It looked like memory leak in the beginning. Indeed, memory consumption increased dramatically from 400M (on 0.6) to over 2G (on 1.1) resulting in OOM under the pressure, since we had a bunch of such services spinning there. After some investigation I've figured out that it wasn’t really memory leak, but rather increased memory usage due to introduction of ThreadPoolExecutor. For instance, ThreadPoolExecutor was configured with 120 threads on our prod server with 24 CPUs, each consuming some amount of memory. Decreasing that workers number somewhat (patching the motor code) I managed to decrease memory usage which approached some value and stayed there with some fluctuations for quite long period of time (just to be 100% sure it’s not memory leak). In our case we didn't need that amount of 120 threads running and we could reduce safely that number without any performance penalty.

      So I was just thinking what if motor had a possibility to tune that workers number? For scenarios like ours. As a possible implementation - via environment variable, let's say MOTOR_THREAD_POOL_EXECUTOR_WORKERS. If that var exists and valid - use that number, otherwise drop to default cpu_count * 5.

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            afonichev Yevhen Afonichev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: