Investigate performance benefit of batching & sorting side-write drain batches

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Similar to SERVER-132351, we should investigate if there are performance benefits to be gained from batching and sorting side write drains.

      In SideWritesTracker::drainWritesIntoIndex(..), side writes are drained in batches, however they are inserted, key-by-key, into the (index) container in SortedDataIndexAccessMethod::insertKeys(..). This last method is used not just by side write drains, however we should investigate the performance benefit of pre-sorting the batch of index keys to be inserted into the index and passing them as a batch to WiredTiger to potentially take advantage of the performance benefits of sorted inserts (e.g. pre-fetching).

      Relationship to the side-write drain decision document

      This ticket is the exploratory work for Option 2 (pre-sort side write drain batches) of the "Draining Side Writes on Disagg Index Builds" decision document, and its result should feed that decision. Beyond the general performance question above, the specific question the decision needs answered is whether pre-sorting can materially improve the drain rate, since a drain rate below the collection's concurrent write rate is what causes an index build to fail to converge at all (SERVER-128241).

      Ordering constraint to resolve

      Side-write entries are applied in strict insertion order today. The side-writes table is keyed by monotonically reserved RecordIds (side_writes_tracker.cpp:87) and drained records are deleted as they are applied, so the forward cursor naturally resumes at the first undeleted record. That ordering is load-bearing: index_build_interceptor.cpp:227 states that documents in this table "must be consumed in insert-order", because a given index key's insert and delete operations must be applied in the order they occurred.

      The investigation therefore needs to establish:

      • Whether sorting within a single drain batch can preserve the relative order of operations on the same key, or whether sorting requires a different conflict-resolution scheme.
      • Whether the locality benefit is real in practice, given that side-write entries are index keys that may be spread across the entire B-tree, which limits how much a batch-local sort can help.

      For reference, batch boundaries are set by maxIndexBuildDrainBatchSize (default 1000 keys) and maxIndexBuildDrainMemoryUsageMegabytes (default 16 MB), declared in index_build_interceptor.idl:12 and :27, with the byte cutoff applied at side_writes_tracker.cpp:284 and the count cutoff at :321.

            Assignee:
            Unassigned
            Reporter:
            Alex Sarkesian
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: