SpoolConsumerStage should be able to retrieve spilled data

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      SpoolConsumerStage has access to a buffer and returns data read from it. The buffer is populated by one of the spool producer stages and it is possible that it has spilled to disk.

      SpoolConsumerStage should have a method that will allow it to retrieve data from a spilled buffer. The stage should make sure to keep the memory usage below the maximum allowed memory threshold at any moment.
      The maximum allowed memory, is defined in internalQueryMaxSpoolDiskUsageBytes query knob.

            Assignee:
            Unassigned
            Reporter:
            Foteini Alvanaki
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: