Use strict comparison to choose final CE result in estimate(OIL) in cardinality estimator

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • Fully Compatible
    • ALL
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Another similar instance of the issue from AF-16798, this time using division instead of subtraction. When estimating index bounds we call estimate(OIL) and use that result in a division operation here, but in estimate(OIL) the CE value returned is chosen by taking the approximate min between the calculated cardinality and _inputCard , which can trip the tassert in the division operator if the value returned is actually larger than _inputCard

      The fix is the same as in SERVER-125501, use strict comparison instead of the approximate comparison operators.

            Assignee:
            Natalie Hill
            Reporter:
            Natalie Hill
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: