[SERVER-40900] Agg pipeline static analysis does not always fail $meta when metadata is unavailable Created: 29/Apr/19  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Optimization
Participants:

 Description   

The aggregation framework has code which makes an effort to fail on "textScore" $meta projection if we can prove, before ever running the query, that "textScore" metadata will never be available. For example, the following query is illegal because the $project with $meta:"textScore" comes after a $match stage which has no $text predicate:

MongoDB Enterprise > db.c.aggregate([{$match: {foo: 1}}, {$project: {out: {$meta: "textScore"}}}])
2019-04-29T17:16:54.642-0400 E  QUERY    [js] uncaught exception: Error: command failed: {
	"ok" : 0,
	"errmsg" : "pipeline requires text score metadata, but there is no text score available",
	"code" : 40218,
	"codeName" : "Location40218"
} : aggregate failed :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
doassert@src/mongo/shell/assert.js:18:14
_assertCommandWorked@src/mongo/shell/assert.js:584:17
assert.commandWorked@src/mongo/shell/assert.js:674:16
DB.prototype._runAggregate@src/mongo/shell/db.js:260:9
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1005:12
@(shell):1:1

This check is made opportunistically during Pipeline dependency analysis. That is, there are cases in which we could easily prove that an illegal $meta refers to "textScore" metadata which is unavailable, but we make no attempt to do so because this is outside the mandate of dependency analysis. For example, we do not perform dependency analysis if the initial stage is an agg metadata source. A $indexStats pipeline with a subsequent "textScore" meta projection could fail during static analysis, but does not:

MongoDB Enterprise > db.c.aggregate([{$indexStats: {}}, {$project: {out: {$meta: "textScore"}}}])
{  } // The "out" field does not appear since there is no "textScore" metadata.

Another example is a "textScore" meta projection following a $group. This could fail, but does not, since dependency analysis does not proceed past $group stages (since subsequent stages after the $group cannot affect the dependency set):

MongoDB Enterprise > db.c.aggregate([{$match: {$text: {$search: "foo bar"}}}, {$project: {b: {$meta: "textScore"}}}, {$group: {_id: null, a: {$addToSet: "$b"}}}, {$project: {a: 1, out: {$meta: "textScore"}}}])
{ "_id" : null, "a" : [ 1.5, 1.1 ] }


Generated at Thu Feb 08 04:56:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.