Match pushdown not attempted before $project hoisting

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • Fully Compatible
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      We should try to pushdown an eligible match before applying the HOIST_SINGLE_DOC_TRANSFORMATION rewrite.

      The code already sets a lower priority for HOIST_SINGLE_DOC_TRANSFORMATION than we have for MATCH_PUSHDOWN, but since the priorities apply only for a specific rewrite position, and not globally, the hoisting (targetting DocumentSourceSingleDocumentTransformation) still applies before the match pushdown (targetting DocumentSourceMatch). 

      This causes an unexpected rewrite, when the complex rename match pushdown is also eligible:

      {$lookup: {as: 'product'}},
      {$unwind: '$product' },
      {$project: {category: '$product.category', country: '$address.country'}},
      {$match: { category: 'Electronics', country: 'Ireland' } 

      In the above case, we will run HOIST_SINGLE_DOC_TRANSFORMATION which will split the complex rename into $addFields, to allow $match pushdown for country even without arrayness information.

      But if we have an index on address.country, the match pushdown will also succeed resulting in:

      {$match: {'address.country': 'Ireland'}} (absorbed into $cursor)
      {$addFields: {country: '$address.country'}},
      {$lookup: {as: 'product'}},
      {$unwind: '$product' },
      {$project: {category: '$product.category'},
      {$match: { category: 'Electronics' }

      Fix

      The fix appears simple - use the same workaround as we do for PUSH_MATCH_BEFORE_UNPACK_BUCKET, by registering a rule targetting DocumentSourceSingleDocumentTransformation which attempts the match pushdown first.

      This ensures the priorities work as intended, avoiding over-eager hoisting for complex renames, when match pushdown can rename the match in the presence of arrayness information indicating the rename does not require array traversal.

            Assignee:
            Vesko Karaganev
            Reporter:
            Vesko Karaganev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: