Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-65884

$lookup from time-series can place $sequentialCache after correlated $match

    XMLWordPrintableJSON

Details

    • Fully Compatible
    • ALL
    • v6.0, v5.3, v5.0
    • QO 2022-05-16, QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QO 2022-07-11, QO 2022-07-25
    • 205

    Description

      When $lookup's foreign collection is time-series, and the inner pipeline has a correlated $match stage, it can incorrectly place a $sequentialCache after the $match.

      For example, suppose we start with:

      db.someCollection.aggregate([
         {$lookup: {
          from: 'timeseriesColl',
          let: {myJoinVar: ...},
          pipeline: [
            {$match: ... myJoinVar ...}
          ]
        }}
      ])
      

      When $lookup optimizes a pipeline, it first resolves the views and tentatively adds a $sequentialCache stage.

      {$_internalUnpackBucket: ...}
      {$match ... myJoinVar ...}
      {$sequentialCache ...}
      

      Then it calls optimizeAt left-to-right. Since the $match does not depend on any measurement fields in the inner pipeline, it correctly swaps with $_internalUnpackBucket:

      {$match ... myJoinVar ...}
      {$_internalUnpackBucket: ...}
      {$sequentialCache ...}
      

      Next we optimize the $_internalUnpackBucket stage. Like $lookup, this stage does something special by calling optimize recursively in its own optimize method. It zooms in on the pipeline suffix:

      {$sequentialCache ...}
      

      Now we optimize $sequentialCache, and it doesn't see the other stages, so it thinks this is an uncorrelated pipeline. It does not remove itself.

      Once all the calls to optimize return we're left with this incorrect pipeline:

      {$match ... myJoinVar ...}
      {$_internalUnpackBucket: ...}
      {$sequentialCache ...}
      

      We should have either removed the $sequentialCache, or positioned it before the correlated $match stage.

      Attachments

        Activity

          People

            ruslan.abdulkhalikov@mongodb.com Ruslan Abdulkhalikov (Inactive)
            david.percy@mongodb.com David Percy
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: