[SERVER-85079] POC: Rewind functionality in document source machinery Created: 22/Mar/21  Updated: 12/Jan/24  Resolved: 28/Apr/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Nicholas Zolnierz Assignee: Ted Tuckman
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Query Optimization 2021-04-05, Query Optimization 2021-04-19, Query Optimization 2021-05-03
Participants:

 Description   

For window functions, in some cases we may want a "two-pass" execution approach such that a first pass computes the result and the second pass projects it to the desired document. This will prevent the need to cache all documents in a partition before streaming. Things to consider:

  • Add "reset" functionality to DS machinery, such that on the first pass we compute the accumulation result over the entire partition then seek back to the first document to project the new field. Would need to reset to the partition boundary as well.
  • Could re-use TeeBuffer and instead of pausing add functionality to spill to disk
  • Spool operator/stage?


 Comments   
Comment by Ted Tuckman [ 28/Apr/21 ]

We decided to go with the approach outlined in WRITING-7751, so closing this.

Generated at Thu Feb 08 06:56:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.