Join optimization: extend the coverage of the plan stability tests

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The first generated plan stability test for join optimization (https://github.com/10gen/mongo/pull/49764) uncovered certain issues in testing coverage that should be addressed before a second generated test is added.

       

      1. Only 5% of the queries return rows
      2. A lot of the queries have a join-optimized SBE prefix, followed by one or more $lookup stages.  This seems to be because the grammar is adding a $match stage in the main pipeline that references fields from the joined collections, which is not a currently-supported scenario
      3. Index access methods are not very varied, there are only a few index lookup ranges, multi-field lookup, multi-range lookups and the like
      4. The clustering algorithm will run until each feature is represented at least once. This may be too little for important features – this should be configurable so that each feature is provably represented in ~10 of queries (if 10 queries from the candidate set had the feature to start with)
      5. OR stages in the plan are underrepresented
      6. Many of the queries have identical or almost-identical counters, which tells me that it is the same join order repeated over and over again with predicates of similar selectivity
      7. Due to the use of top-level $match stages, many of the $lookups are not join-optimization-eligible and therefore remain at the end of the join order
      8. Sarg-able predicates are few and far between, they need to be weighted more than non-sargable predicates.
      9. Large $in lists are not generated

            Assignee:
            Philip Stoev
            Reporter:
            Philip Stoev
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: