Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-40900

Agg pipeline static analysis does not always fail $meta when metadata is unavailable

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
      None
    • Query Optimization

      The aggregation framework has code which makes an effort to fail on "textScore" $meta projection if we can prove, before ever running the query, that "textScore" metadata will never be available. For example, the following query is illegal because the $project with $meta:"textScore" comes after a $match stage which has no $text predicate:

      MongoDB Enterprise > db.c.aggregate([{$match: {foo: 1}}, {$project: {out: {$meta: "textScore"}}}])
      2019-04-29T17:16:54.642-0400 E  QUERY    [js] uncaught exception: Error: command failed: {
      	"ok" : 0,
      	"errmsg" : "pipeline requires text score metadata, but there is no text score available",
      	"code" : 40218,
      	"codeName" : "Location40218"
      } : aggregate failed :
      _getErrorWithCode@src/mongo/shell/utils.js:25:13
      doassert@src/mongo/shell/assert.js:18:14
      _assertCommandWorked@src/mongo/shell/assert.js:584:17
      assert.commandWorked@src/mongo/shell/assert.js:674:16
      DB.prototype._runAggregate@src/mongo/shell/db.js:260:9
      DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1005:12
      @(shell):1:1
      

      This check is made opportunistically during Pipeline dependency analysis. That is, there are cases in which we could easily prove that an illegal $meta refers to "textScore" metadata which is unavailable, but we make no attempt to do so because this is outside the mandate of dependency analysis. For example, we do not perform dependency analysis if the initial stage is an agg metadata source. A $indexStats pipeline with a subsequent "textScore" meta projection could fail during static analysis, but does not:

      MongoDB Enterprise > db.c.aggregate([{$indexStats: {}}, {$project: {out: {$meta: "textScore"}}}])
      {  } // The "out" field does not appear since there is no "textScore" metadata.
      

      Another example is a "textScore" meta projection following a $group. This could fail, but does not, since dependency analysis does not proceed past $group stages (since subsequent stages after the $group cannot affect the dependency set):

      MongoDB Enterprise > db.c.aggregate([{$match: {$text: {$search: "foo bar"}}}, {$project: {b: {$meta: "textScore"}}}, {$group: {_id: null, a: {$addToSet: "$b"}}}, {$project: {a: 1, out: {$meta: "textScore"}}}])
      { "_id" : null, "a" : [ 1.5, 1.1 ] }
      

            Assignee:
            backlog-query-optimization [DO NOT USE] Backlog - Query Optimization
            Reporter:
            david.storch@mongodb.com David Storch
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: