The MongoDB query optimizer uses an empirical plan ranking strategy in which candidate plans are executed in order to determine which is likely to be most efficient. The trial period ends when either
- a plan executes the entire query,
- a plan generates an entire batch of results, or
- some large maximum amount of work has been done.
In the third case, it is possible that the trial period will be expensive, since there could be n candidate plans each performing a significant amount of IO or database operations.
If the candidates execute for a long time—say, 10,000 units of work—without hitting end condition #1 or end condition #2, we should pause and eliminate any candidate plan that has already proven to be suboptimal. This same progressive refinement strategy could be done every w units of work in order to minimize the cost of the plan selection trial period.