There are cases where CBR with sampling CE picks one plan >>0% and <<100% of the time.

Such cases frequently cause noise in performance analyses, when a set of runs all choose one plan, and a run in the other set chooses a different plan, leading to apparent improvements or regressions.

This is time consuming to manually analyse, so we should have a tool that can automatically analyse the probability distribution of CBR with sampling CE choosing different plans. If this distribution over plans is statistically indistinguishable before and after a patch that affects plan costing, we can consider that task's performance unchanged.