Add spilling in DocumentSourceSequentialDocumentCache

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      DocumentSourceSequentialDocumentCache caches documents in lookup pipelines that do not use local/foreign field match.  If the size of the cached document exceeds a limit the stage flushes everything and instead executes the pipeline for every iteration. We should investigate whether it is beneficial for this stage to spill instead of executing the query again and again.
       
      Example lookup that uses the DocumentSourceSequentialDocumentCache
       

      {
          $lookup: {
            from: "customers",
            let: { orderCustomerId: "$customerId" },
            pipeline: [
              {
                $match: {
                  $expr: {
                    $eq: ["$_id", "$$orderCustomerId"]
                  }
                }
              }
            ],
            as: "customerDetails"
          }
      } 

            Assignee:
            Unassigned
            Reporter:
            Foteini Alvanaki
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: