Thread SimpleMemoryUsageToken through InternalSetWindowFieldsStage

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      InternalSetWindowFieldsStage already has a tracker to track its permanent state memory usage. This tracker should be threaded to all evaluate() calls. More specifically, InternalSetWindowFieldsStage calls evaluate() in three cases:

      1. When it evaluates expressions as part of windows. Each window function inherits from WindowFunctionExec, which already receives its own tracker (functionMemTracker, created in WindowFunctionExec::create() via (*memTracker)[fieldName]) and stores it as the member variable _memTracker. This can be used by all window functions that call evaluate().
      2. During initialisation of N expressions ($firstN/$lastN/$maxN/etc.), in order to evaluate the value of N. This uses the same per-field tracker as case 1 (functionMemTracker). The tracker needs to be threaded from WindowFunctionExec::create() into buildAccumulatorOnly() and buildRemovable(), which then pass it to evaluate().
      3. For the evaluation of the partitionBy expression. PartitionIterator already holds a stage-wide sub-tracker, obtained via _memoryToken, keyed by the fixed string "PartitionIterator" (via (*tracker)["PartitionIterator"]). _memoryToken.tracker() should be threaded into PartitionKeyComparator's constructor and used in its evaluate() call.

      The flags gFeatureFlagExpressionMemoryTracking and gFeatureFlagMemoryTracking must be checked before passing the tracker.

            Assignee:
            Unassigned
            Reporter:
            Foteini Alvanaki
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: