ExportXMLWordPrintableJSON

    • Type: Investigation
    • Resolution: Fix Not Needed
    • Priority: Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • None
    • Not Needed
    • Developer Tools

      Original Downstream Change Summary

      Previously, if $geoNear's query predicate named a field that the geo index did not cover, the planner could not assign it to the index scan and left it on a FETCH stage placed above the GEO_NEAR stage.

      This change adds a planner pass that can collapse a {{FETCH

      {filter} -> GEO_NEAR}} pair into the geo node and evaluates the residual predicate inside the geo stage's per-interval fetch, before any downstream work such as document materialization or distance sorting happens for documents that would later be filtered out anyway.

      Due to this optimization, the explain output for affected queries can change its shape:
      the FETCH above GEO_NEAR disappears and a residualFilter field appears on the geo node, containing the residual (non-geo index-specific) filter predicate. The index filter is still returned as is inside the filter field.

      There is no change to which documents are returned. The filter still sees the exact same set of candidate documents it would have seen before; however, the residual predicate is applied earlier in the pipeline. Result order and contents stay the same.
      However, the change introduces a potential small behavior change w.r.t. throwing residual filter predicates.
      The resdiual filter is now applied on candidate documents directly after the distance checks (minDistance and maxDistance), but before the candidate document is buffered, sorted, and fully owned.
      A throwing residual filter predicate still can only throw on a document that is a legitimate candidate (document inside the search annulus) for the result set, but it can do so now earlier than before.
      A query using a limit can now apply a throwing residual predicate before the limit is satisfied. Such queries may now fail if the residual filter throws on a candidate document, even though they could previously work (by chance). The old order in which documents were evaluated and run through the filtering was never precisely guaranteed. Also the geo stage can (and already did) over-fetch documents even when a limit is set, which can lead to the residual filter being executed on more documents than prescribed by the limit itself.

      The optimization is only applied for the query predicate of geo queries. It is currently not applied for geo queries following by a trailing $match. The trailing match is currently not auto-collapsed into the geo stage, so the optimization is not triggered.
      h2. Description of Linked Ticket
      `$geoNear`'s query predicate is merged into the find filter by `DocumentSourceGeoNear::asNearQuery()`. When it names a field the geo index doesn't cover, the planner cannot assign it to the index scan, so `buildIndexedAnd()` leaves it on a FETCH stage placed above GEO_NEAR.

      That means every document the near stage returns pays for distance computation, BSON ownership and a round trip through the BoundedSorter, and only then gets discarded by a predicate that could have run first. On the `Aggregation.GeoNear2d` mongo-perf workload roughly half the buffered documents were thrown away this way: the stage buffered 988 documents to return 224.

      This patch adds a planner pass, `pushResidualFilterIntoGeoNear()`, that collapses a `FETCH{filter}

      then GEO_NEAR` pair into the geo node and evaluates the predicate in the stage's per-interval fetch, before any of that downstream work. It is result-equivalent — the same MatchExpression, against the same documents, evaluated lower in the tree. Removing the FETCH is legal because `GeoNear*Node::fetched()` is already true and the node reports all fields as provided. For the 2D path the annulus check is kept first in the AND, since it is cheap and geo-selective.

      The predicate goes into a new `docFilter` field rather than the existing `filter`. GeoNear2DStage routes `GeoNearParams::filter` to its covered IndexScan as a key-level filter, whose IndexKeyMatchableDocument throws massert 17409 for any field absent from the key pattern. Reusing `filter` would turn this into a user-visible query failure on 2D.

      `removeImpreciseInternalExprFilters()` now also learns about `docFilter`: it strips imprecise `$_internalExpr*` conjuncts from `fetched()` nodes, and without this the pushdown would have smuggled a redundant conjunct past it and evaluated it per document.

      Nothing here crosses a process boundary: QuerySolution is never serialized, mongos ships commands rather than plans, and the plan cache stores index tags and rebuilds solutions through `analyzeDataAccess()` on every hit. Mixed-version clusters need no gating. Explain output does change shape — the FETCH above GEO_NEAR disappears and a `docFilter` appears on the geo node. This is likely a lesser concern.

      Measurements (local LTO+PGO build, no BOLT; mongo-perf, 1 thread, n=9–11 paired reps):
      ```
      ┌─────────────────────────────┬──────────┬─────────────┐
      │ test │ instr/op │ ops/sec │
      ├─────────────────────────────┼──────────┼─────────────┤
      │ Aggregation.GeoNear2d │ −17.6% │ +24% … +34% │
      ├─────────────────────────────┼──────────┼─────────────┤
      │ Aggregation.GeoNear2dSphere │ −21.6% │ +31% … +45% │
      └─────────────────────────────┴──────────┴─────────────┘
      ```

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: