• Type: Bug
    • Resolution: Unresolved
    • Priority: Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Found by an AI bug hunt and confirmed by a differential oracle: results differ with internalQueryPlannerPushdownFilterToIxscanForSort on and off, while the classic and SBE engines agree with each other.

      Problem

      On a collection with a non-simple collation, a string equality filter that the planner pushes into a full index scan (used only to provide the sort) matches nothing. The index scan's filter is evaluated against the index keys, which hold collation keys for strings, not the strings. Matching documents are dropped before the fetch. Both engines return the same wrong result. The knob internalQueryPlannerPushdownFilterToIxscanForSort is on by default.

      Repro

      Fails at the local build of 2026-09-17; no later commit changes the filter pushdown. The collection has collation locale "en", strength 2, and an index on b then a. The query matches a equal to "abc", sorts by b and projects _id. The documents with _id 1 and 2 hold "abc" and "ABC".

      Run Result (expected _id 1, 2)
      pushdown on, classic engine no documents
      pushdown on, SBE no documents
      pushdown off, either engine 1, 2
      collection scan ($natural hint) 1, 2

      With the pushdown on, the IXSCAN stage in the explain carries the filter shown below. Self-contained jstest; controls first, the failing assertion last:

      // IXSCAN filter in the explain with the pushdown on: { "a" : { "$eq" : "abc" } }
      /**
       * A filter pushed into a full index scan that exists only to provide a sort compares strings
       * against the index's collation keys on a collated collection, so matching documents are dropped.
       * The result is the same wrong answer in the classic and SBE engines.
       */
      const conn = MongoRunner.runMongod({setParameter: {internalQueryDisablePlanCache: true}});
      const db = conn.getDB(jsTestName());
      assert.commandWorked(db.createCollection("c", {collation: {locale: "en", strength: 2}}));
      assert.commandWorked(db.c.insertMany([
          {_id: 1, b: 1, a: "abc"},
          {_id: 2, b: 2, a: "ABC"},
          {_id: 3, b: 3, a: "xyz"},
          {_id: 4, b: 4},
      ]));
      assert.commandWorked(db.c.createIndex({b: 1, a: 1}));
      
      const pipeline = [{$match: {a: "abc"}}, {$sort: {b: 1}}, {$project: {_id: 1}}];
      const ids = (docs) => docs.map((d) => d._id);
      function ixscanFilters(explain) {
          const found = [];
          (function walk(x) {
              if (Array.isArray(x)) return x.forEach(walk);
              if (x === null || typeof x !== "object") return;
              if (x.stage === "IXSCAN") found.push(x.filter);
              Object.values(x).forEach(walk);
          })(explain);
          return found;
      }
      function run(label, engine, pushdown, hint) {
          assert.commandWorked(db.adminCommand({
              setParameter: 1,
              internalQueryFrameworkControl: engine,
              internalQueryPlannerPushdownFilterToIxscanForSort: pushdown,
          }));
          const opts = hint ? {hint} : {};
          const filters = ixscanFilters(db.c.explain().aggregate(pipeline, opts));
          const got = ids(db.c.aggregate(pipeline, opts).toArray());
          jsTest.log.info("run", {label, engine, pushdown, hint, ixscanFilters: filters, got});
          return {got, filters};
      }
      
      // Controls: the answer is [1, 2] when the filter is not pushed into the index scan or when the
      // query scans the collection.
      assert.eq(run("pushdown off", "forceClassicEngine", false).got, [1, 2], "pushdown off, classic");
      assert.eq(run("pushdown off", "trySbeEngine", false).got, [1, 2], "pushdown off, SBE");
      assert.eq(run("collection scan", "forceClassicEngine", true, {$natural: 1}).got, [1, 2], "collscan");
      // Mechanism: with the pushdown on, the filter is in the index scan.
      assert(run("pushdown on", "forceClassicEngine", true).filters.some((f) => f !== undefined),
             "the filter was not pushed into the index scan; the repro does not test the bug");
      // Fails at the local build (2026-09-17): returns [] in both engines.
      assert.eq(run("pushdown on", "forceClassicEngine", true).got, [1, 2], "pushdown on, classic");
      assert.eq(run("pushdown on", "trySbeEngine", true).got, [1, 2], "pushdown on, SBE");
      MongoRunner.stopMongod(conn);
      

      Cause (from code reading, not verified)

      • pushdownFilterToFullIxscan() in src/mongo/db/query/planner_access.cpp keeps every indexed leaf in the index scan filter except $exists and $type: null, with no check of the index collation or of bounds tightness.
      • A string predicate counts as index-compatible when the query and index collations match (planner_ixselect.cpp). The normal covered-filter path marks strings on collated indexes as needing a fetch; the pushdown path does not.
      • The SBE filter over an index scan reads the key slot directly (gen_index_scan.cpp, gen_filter.cpp).

      Impact

      Silent wrong results with default settings for a string predicate on a collated collection when the planner chooses a full index scan to provide a sort.

      Related

      • SERVER-103601: introduced the pushdown for full index scans.
      • SERVER-125720: an earlier bug in the same knob ($expr with dotted paths), found by a property test. index_for_sort_pbt.js does not use collations.

            Assignee:
            Unassigned
            Reporter:
            Zixuan Zhuang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: