Unite assumptions about shard filter node in CBR

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      CBR makes the same assumption about STAGE_SHARDING_FILTER in two places. The assumption is that chunk migrations are rare and thus the sharding filter acts as a passthrough node (similar to projections), where its selectivity is 1.
      1. CardinalityEstimator::estimate() routes it through passThroughNodeCard
      2. CardinalityEstimator::propagateLimit() lists it alongside the projection stages and propagates the limit to the child unchanged

      If we ever refine shard filter CE so that it models actual filtering (i.e. SERVER-129663), the limit propagation will not be in line with that and its easy to miss.

      One idea for this is to modify the limit propagation code such that the passthrough stages get put with the fetch stage here. This should work because looking at where limitFraction is defined:

      • we set the FETCH's children CE to be limitFraction * inCE
      • limitFraction * inCE = (limitCE / outCE) * inCE
      • for the passthrough nodes, outCE = inCE, so (limitCE / outCE) * inCE = limitCE

      So the effect should be the same as what we do now for propagating the limit to the children of the passthrough nodes. But we should test this carefully to ensure we don't get any issues with rounding here.

            Assignee:
            Unassigned
            Reporter:
            Militsa Sotirova
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: