Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44984

Reduce index thread pool size and reduce memory used per build

    XMLWordPrintableJSON

Details

    • Fully Compatible
    • Execution Team 2019-12-16, Execution Team 2020-01-13, Execution Team 2020-01-27, Execution Team 2020-02-10, Execution Team 2019-12-30, Execution Team 2020-02-24, Execution Team 2020-03-09
    • 16

    Description

      The index build thread pool size, which is used by index builds on secondary nodes in 4.2 and for index builds on all nodes in 4.4+, was reduced to 3. This means that for a set of indexes for a collection, built simultaneously via one createIndexes command, each of these is limited to 3 concurrently running on secondary nodes in 4.2 and on all nodes in 4.4. Any index build sets in excess of three will block until another index build set finishes.

      In addition, the default memory cap used to build an index set was reduced from 500MB to 200MB. This memory cap governs how often the external sorter spills to disk.

      Original Description: The thread pool size is currently set to 10, but with today's default setting of 500MB for each external sorter, this limit makes it possible to exceed most instances' free memory. If we reduced the limit to something like 3, that would increase stability, without sacrificing functionality.

      Attachments

        Issue Links

          Activity

            People

              milkie@mongodb.com Eric Milkie
              milkie@mongodb.com Eric Milkie
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: