[Join Optimization] Avoid cache invalidation when a relevant but unused index is dropped

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      After SERVER-130368, the join plan cache will use index fingerprinting of relevant indexes to determine whether a plan cache entry needs to be invalidated. If we see that the collection version tags, differ, the following states of index fingerprinting will be used to determine whether or not to replan:

      1. Index that we considered but didn't use in plan is dropped/modified -> invalidate
      2. Index that we used in plan is dropped/modified -> invalidate
      3. Index added that could now be used in plan -> invalidate
      4. Index that we didn't consider is dropped/modified -> reuse plan

      This ticket should implement logic such that (1) doesn't invalidate the cache.

      This will require us to walk over the final QuerySolution that is produced when we initially plan to determine the set of indexes that were used per node and store that in the plan cache entry.

      Later, when we try to pull the plan from cache and see a collection version mismatch, we should check if the indexes used by a node were modified or not.

      One sketch of the solution could be:

      1. Modify what we store in the plan cache to be: used index fingerprint, relevant index fingerprint and relevant index count
      2. When reconstructing the QuerySolution from the cache we find all the indexes used by the nodes.
      3. Lastly, we would need to perform the following checks if there is a collection version mismatch:
      if (cached.usedFingerprint != current.usedFingerprint) -> replan
      if (current.relevantIndexCount > cached.relevantIndexCount) -> replan
      if (current.relevantIndexCount < cached.relevantIndexCount) -> reuse (removal of index that wasn't used
      if (cached.relevantFingerprint != current.relevantFingerprint) -> replan
      

            Assignee:
            Naafiyan Ahmed
            Reporter:
            Naafiyan Ahmed
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: