Investigate incremental spilling in primary-driven index builds

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      During index builds, the sorter can accumulate significant amount of index keys in memory (default 200MB) before spilling, resulting in a workload where we have large bursts of writes that can be fairly spaced out. This can interact poorly with flow control, which can kick in due to standby replication lag.

      We should investigate if we can smooth out this workload with some form of incremental spilling, with the goal of getting better performance by avoiding flow control kicking in due to standby replication lag.

            Assignee:
            Thomas Goyne
            Reporter:
            Alex Sarkesian
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: