Rate limit the writes performed during the bulk load phase of primary-driven index builds

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • Fully Compatible
    • Storage Execution 2026-08-31
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Add a server parameter to throttle the rate at which the bulk load phase of a primary-driven index build writes accumulated sorted keys from the sorter into the index, for the same reason as the scan phase throttle.

      As with the scan phase, the target is the write rate – here that is the replicated container writes generated as keys are loaded into the index table.

      The two phases have very different write profiles and need independent limits, so this is a second parameter rather than a shared one.

      Implementation notes

      • Reuse DataThrottle (src/mongo/db/throttle_cursor.h:132) as in the scan phase ticket, charging bytes written.
      • Throttle at the batch boundary in BulkBuilderImpl::commit (src/mongo/db/index/index_access_method.cpp:1167) – after wunit.commit() in the commitBatch lambda (:1207-1234), charging bytesInBatch. The per-key work inside the unit of work is _addKeyForCommit (:1216, implementation at :1376); the sleep must be outside the WriteUnitOfWork, not between keys.
      • The existing yieldFn (multi_index_block.cpp:1198) is the safe place to sleep with locks released.
      • Driven from MultiIndexBlock::dumpInsertsFromBulk (multi_index_block.cpp:1133, per-index loop at :1238-1273), where the primary-driven batch size/bytes only matter when _containerWriteBehavior == kReplicate. Scope the throttle to that same condition.
      • Foreground index builds must be unaffected. The new parameter belongs in src/mongo/db/index_builds/primary_driven_index_build_knobs.idl.

            Assignee:
            Damian Wasilewicz
            Reporter:
            Alex Sarkesian
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: