Implement DISTINCT_N_SCAN

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Today, we already leverage distinct scans for $group + $first/$last/$bottom/$top. We can do the same for accumulators like $topN even when N > 1. We prototyped this optimization in SKUNK-30 and saw huge improvements on Query High Value Workloads. When eligible, this is an optimization we should always do.

      Required work:

      • Extend distinct scan execution to accept parameter N
      • Ensure $groupByDistinctScan works for cases where N > 1
      • Extend planning logic to pick DISTINCT_N_SCAN in eligible cases

            Assignee:
            Unassigned
            Reporter:
            Max Verbinnen
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: