Memory used by sorter spills can grow without bound

XMLWordPrintableJSON

    • Query Execution
    • Fully Compatible
    • ALL
    • v8.0, v7.0, v6.0
    • QE 2024-04-29, QE 2024-05-13, QE 2024-05-27, QE 2024-06-10
    • 200
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      A sorter maintains a vector of "iterators" for each range of data that it spills to disk. Each allocation of this FileIterator is 128 bytes, plus the shared_ptr's control block. Thus with enough spills, the amount of memory being used can easily exceed the value configured by maxIndexBuildMemoryUsageMegabytes. This can also be exacerbated by building multiple indexes in a single index build, since the memory limit will be shared among the indexes and thus each one will spill more often.

              Assignee:
              Foteini Alvanaki
              Reporter:
              Gregory Noma
              Votes:
              0 Vote for this issue
              Watchers:
              25 Start watching this issue

                Created:
                Updated:
                Resolved: