Add $sort pushdown optimization to $_internalSearchIdLookup

XMLWordPrintableJSON

    • Query Integration
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      If a $sort (on metadata fields) stage is present directly after an $_internalSearchIdLookup stage, the $sort stage can be safely pushed down before the $_internalSearchIdLookup. Conceptually, it should not matter if the results are sorted before or after they are converted from id's to full documents, as long as the $sort criteria is on metadata (because the metadata fields should all be present before lookup).

       

      We want to push down the $sort when possible, because other stages before $_internalSearchIdLookup maybe be able to optimize it away. For example, a $search or $vectorSearch, whose results are sorted on the metadata 'searchScore' or 'vectorSearchScore', are able to remove a $sort that is sorted on the same metadata criteria.

       

      Make sure to add test cases to pipeline_test.cpp, that manually add a $_internalSearchIdLookup stage before a $sort on metadata, then check that after pipeline optimization, the $sort appears before the $_internalSearchIdLookup.

              Assignee:
              Unassigned
              Reporter:
              Joe Shalabi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: