-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Query Optimization 2021-04-05, Query Optimization 2021-04-19, Query Optimization 2021-05-03
For window functions, in some cases we may want a "two-pass" execution approach such that a first pass computes the result and the second pass projects it to the desired document. This will prevent the need to cache all documents in a partition before streaming. Things to consider:
- Add "reset" functionality to DS machinery, such that on the first pass we compute the accumulation result over the entire partition then seek back to the first document to project the new field. Would need to reset to the partition boundary as well.
- Could re-use TeeBuffer and instead of pausing add functionality to spill to disk
- Spool operator/stage?