Add generic $sort optimization to erase redundant $sorts

XMLWordPrintableJSON

    • Query Integration
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      will.buerger@mongodb.com suggested adding a generic $sort optimization (via a rule that lives on the $sort stage) that erases itself (the $sort stage) when it sorts by the latest sort pattern. This ticket should investigate the feasibility of this and based on that determination, implement it/add the appropriate unit/integration tests.

      More details excerpted from Will here:
      "In other words, if stages are responsible for reporting metadata about document sort-key population, then we can register an optimization with the $sort stage that introspects the sort-key populated once that stage is reached (a la dependency analysis). Related to my comment about making this optimization apply to $search as well, this would make this optimization apply more generically

      That also might help with sharding? Since we're looking behind instead of ahead, we don't need to worry about the shard-side losing pipeline suffix dependencies. $sort ends up in the merging pipeline so it only requires analysis pre-pipeline-split"

      This ticket should

      • focus on implementing the rule such that it optimizes away all redundant $sorts on "vectorSearchScore" or "searchScore" metadata for both extension and non-extension search stages.
      • thoroughly test how ext stages interact w the new sort rule. update $testVectorSearchOptimization as needed so that it correctly interfaces with the rule's precondition logic.
      • get rid of the 
        is_stage_sorted_by_vector_search_score_deprecated function as the sort rule will account for the fallback sort opt behavior for $extensionVectorSearch

            Assignee:
            Adithi Raghavan
            Reporter:
            Adithi Raghavan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: