-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Critical - P2
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Optimization
-
ALL
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Found by an AI bug hunt and confirmed by a differential oracle: results differ with internalQueryPlannerPushdownFilterToIxscanForSort on and off, while the classic and SBE engines agree with each other.
Problem
A $expr filter that compares a field with null using $lt, on a field that is not the leading field of an index that provides the sort, is pushed into a full index scan. In aggregation expressions a missing field sorts below null, so the comparison is true for documents without the field. The index key for a missing field is null, and null is not less than null, so those documents are dropped before the fetch can apply the exact $expr. Both engines return the same wrong result. The knob internalQueryPlannerPushdownFilterToIxscanForSort is on by default.
Repro
Fails at the local build of 2026-09-17; no later commit changes the filter pushdown. The index is on c then w. The query is $expr with $lt of $w and null, sorted by c, projecting _id. Documents 1 and 5 have no w, document 2 has w null, and documents 3 and 4 have a number and a string.
| Run | Result (expected _id 1, 5) |
|---|---|
| pushdown on, classic engine | no documents |
| pushdown on, SBE | no documents |
| pushdown off, either engine | 1, 5 |
| collection scan ($natural hint) | 1, 5 |
With the pushdown on, the IXSCAN stage in the explain carries the filter shown below. Self-contained jstest; controls first, the failing assertion last:
// IXSCAN filter in the explain with the pushdown on: { "w" : { "$_internalExprLt" : null } } /** * A {$expr: {$lt: [<field>, null]}} filter pushed into a full index scan that exists only to * provide a sort drops documents where the field is missing: the index key for a missing field is * null and null < null is false, while in aggregation expressions missing sorts below null. * The result is the same wrong answer in the classic and SBE engines. */ const conn = MongoRunner.runMongod({setParameter: {internalQueryDisablePlanCache: true}}); const db = conn.getDB(jsTestName()); assert.commandWorked(db.c.insertMany([ {_id: 1, c: 1}, {_id: 2, c: 2, w: null}, {_id: 3, c: 3, w: 1}, {_id: 4, c: 4, w: "x"}, {_id: 5, c: 5}, ])); assert.commandWorked(db.c.createIndex({c: 1, w: 1})); const pipeline = [{$match: {$expr: {$lt: ["$w", null]}}}, {$sort: {c: 1}}, {$project: {_id: 1}}]; const ids = (docs) => docs.map((d) => d._id); function ixscanFilters(explain) { const found = []; (function walk(x) { if (Array.isArray(x)) return x.forEach(walk); if (x === null || typeof x !== "object") return; if (x.stage === "IXSCAN") found.push(x.filter); Object.values(x).forEach(walk); })(explain); return found; } function run(label, engine, pushdown, hint) { assert.commandWorked(db.adminCommand({ setParameter: 1, internalQueryFrameworkControl: engine, internalQueryPlannerPushdownFilterToIxscanForSort: pushdown, })); const opts = hint ? {hint} : {}; const filters = ixscanFilters(db.c.explain().aggregate(pipeline, opts)); const got = ids(db.c.aggregate(pipeline, opts).toArray()); jsTest.log.info("run", {label, engine, pushdown, hint, ixscanFilters: filters, got}); return {got, filters}; } // Controls: missing < null is true, so documents 1 and 5 match, when the filter is not pushed // into the index scan or when the query scans the collection. assert.eq(run("pushdown off", "forceClassicEngine", false).got, [1, 5], "pushdown off, classic"); assert.eq(run("pushdown off", "trySbeEngine", false).got, [1, 5], "pushdown off, SBE"); assert.eq(run("collection scan", "forceClassicEngine", true, {$natural: 1}).got, [1, 5], "collscan"); // Mechanism: with the pushdown on, the filter is in the index scan. assert(run("pushdown on", "forceClassicEngine", true).filters.some((f) => f !== undefined), "the filter was not pushed into the index scan; the repro does not test the bug"); // Fails at the local build (2026-09-17): returns [] in both engines. assert.eq(run("pushdown on", "forceClassicEngine", true).got, [1, 5], "pushdown on, classic"); assert.eq(run("pushdown on", "trySbeEngine", true).got, [1, 5], "pushdown on, SBE"); MongoRunner.stopMongod(conn);
Cause (from code reading, not verified)
- rewrite_expr.cpp accepts a null right-hand side, so the $expr becomes an internal comparison.
- The pushdown (pushdownFilterToFullIxscan() in src/mongo/db/query/planner_access.cpp) treats only $exists and $type: null as unsafe for the index scan filter.
- For bounded scans index_bounds_builder.cpp widens a null comparison to include missing values and marks the predicate as needing a fetch; the pushdown path has no equivalent guard.
Impact
Silent wrong results with default settings for an $expr null comparison combined with a sort that an index provides.
Related
SERVER-103601: introduced the pushdown for full index scans.SERVER-125720: an earlier bug in the same knob ($expr with dotted paths), found by a property test. index_for_sort_pbt.js avoids missing fields.
- is related to
-
SERVER-136628 Filter pushed into a sort-providing full index scan compares strings against collation keys on a collated collection
-
- Needs Scheduling
-
- related to
-
SERVER-103601 Optimize AND trees for full IXSCAN plans
-
- Closed
-
-
SERVER-125720 `internalQueryPlannerPushdownFilterToIxscanForSort` can cause tassert for $or queries
-
- Closed
-