Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44984

Reduce index thread pool size and reduce memory used per build

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Sprint:
      Execution Team 2019-12-16, Execution Team 2020-01-13, Execution Team 2020-01-27, Execution Team 2020-02-10, Execution Team 2019-12-30, Execution Team 2020-02-24, Execution Team 2020-03-09
    • Linked BF Score:
      16

      Description

      The index build thread pool size, which is used by index builds on secondary nodes in 4.2 and for index builds on all nodes in 4.4+, was reduced to 3. This means that for a set of indexes for a collection, built simultaneously via one createIndexes command, each of these is limited to 3 concurrently running on secondary nodes in 4.2 and on all nodes in 4.4. Any index build sets in excess of three will block until another index build set finishes.

      In addition, the default memory cap used to build an index set was reduced from 500MB to 200MB. This memory cap governs how often the external sorter spills to disk.

      Original Description: The thread pool size is currently set to 10, but with today's default setting of 500MB for each external sorter, this limit makes it possible to exceed most instances' free memory. If we reduced the limit to something like 3, that would increase stability, without sacrificing functionality.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              milkie Eric Milkie
              Reporter:
              milkie Eric Milkie
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: