Index build drain batch sizes are only constrained by number of keys set by maxIndexBuildDrainBatchSize

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      maxIndexBuildDrainBatchSize doesn't necessarily take into account the 16mb oplog entry size limit so we may generate unnecessarily large oplog entries when draining the side writes table at the end of an index build.
       
      We may want to add a different way to constrain drain batch sizes for resumable primary driven index builds to:

      1. Reduce running into large batches requiring chained applyOps that may be split in the oplog. (Although deleting from the side write table comes after so the write to the index so in this case, when the applyOps is split, the worst case is we reapply the write to the index which will only be an insert or delete so that should be idempotent).
      2. Limit the storage transaction size. 

            Assignee:
            Unassigned
            Reporter:
            Shin Yee Tan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: