[CLASSIC] AndHashStage should spill if allowDiskUse is specified and memory usage has exceeded some limit

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      AndHashStage uses an unordered map to store the records from the first child. It tracks the memory this consumes and compares it against a limit defined in a variable inside the cpp file. We will convert this to a query knob and spill the data when the memory exceeds this limit instead of spilling.

       

      Additionally the unordered set it uses to track records already seen should be replaced by a RecordIdDeduplicator and spilled if necessary in order for the memory consumption to drop below the memory usage limit..

              Assignee:
              Unassigned
              Reporter:
              Foteini Alvanaki
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: