$lookup with dotted 'as' can incorrectly push $match before fields are rewritten

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: 8.0.0, 8.3.0
    • Component/s: None
    • Query Optimization
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      When $lookup writes to a dotted as path such as "a.b", it can replace the parent field "a".
      For example, if the input document is:

      {a: [{d: 5}]}
      

      then $lookup with as: "a.b" coerces "a" into:

      {b: []}
      

      This removes "a.d". The optimizer currently treats $lookup as only modifying the exact path "a.b". Because of that, it may incorrectly push a later $match on sibling path "a.d" before the $lookup.
      Example pipeline:

      [
        {$lookup: {from: "joined", as: "a.b", pipeline: []}},
        {$addFields: {"a.c": "$x"}},
        {$match: {"a.d": 5}}
      ]
      

      Correct behavior: $match runs after $lookup, "a.d" no longer exists, and the query returns 0 documents.
      Buggy behavior: $match is pushed before $lookup, sees the original document with "a.d", and incorrectly returns 1 document.

      I reproduced on 8.0, 8.3 and master, but it probably goes back much further.

            Assignee:
            Unassigned
            Reporter:
            Max Verbinnen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: