[SERVER-38716] fix $geoNear and $text within $lookup sub-pipeline Created: 19/Dec/18  Updated: 29/Oct/23  Resolved: 17/Jan/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 3.6.9, 4.0.5, 4.1.7
Fix Version/s: 4.1.8

Type: Bug Priority: Critical - P2
Reporter: Brigitte Lamarche (Inactive) Assignee: Brigitte Lamarche (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File lookup_subpipeline_geonear.js    
Issue Links:
Depends
is depended on by SERVER-38995 Allow geoNear within a lookup pipelin... Closed
is depended on by SERVER-38996 Allow $text within a lookup pipeline ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Query 2019-01-14, Query 2019-01-28
Participants:

 Description   

When a $geoNear stage is part of a $lookup stage's pipeline, this aggregation will fail with error code 50860. Attached is a script which can be used to reproduce this error by running it from the jstests/aggregation/sources/lookup directory: 

 buildscripts/resmoke.py --suites='aggregation' jstests/aggregation/sources/lookup/lookup_subpipeline_geonear.js 

Part of fixing this test will include making sure this works in the sharded case, and adding test coverage for $geoNear within a $lookup subpipeline. This will include testing that this works with $geoNear using a $$ variable within the query option. For example:

  const pipeline = [
        {
          $lookup: {
              let : {var1: "$x"},
              pipeline: [
                  {$geoNear: {near: [0, 0], distanceField: "distance", spherical: true, query: {_id:"$$var1"},}}, 
              ],
              from: "from",
              as: "c",
          }
        },
    ];

 This does not affect 4.0.



 Comments   
Comment by Githook User [ 17/Jan/19 ]

Author:

{'username': 'bLamarche413', 'email': 'thesisiwbl@gmail.com', 'name': 'Brigitte Lamarche'}

Message: SERVER-38716 fix $geoNear and $text within $lookup sub-pipeline
Branch: master
https://github.com/mongodb/mongo/commit/21e65b332ed546077d6e54a4c5d43e02ef9dcbcc

Comment by Asya Kamsky [ 02/Jan/19 ]

The problem I pointed out is happening in 4.0 so it might be a different issue entirely.

Comment by Asya Kamsky [ 02/Jan/19 ]

For $geoNear, when I use {_id:"$$var"} as match expression (query in $geoNear agg stage) it correctly matches documents which have _id value literal string "$$var".  However, when I change query to use $expr to inject a variable, it works as a stand-alone agg stage but not as expressive $lookup.

db.d.aggregate({$geoNear:{near:[0,0], distanceField:"distance", spherical:true, query:{$expr:{$eq:["$_id",{$literal:"$$var"}]}}}})
// works
db.d.aggregate({$geoNear:{near:[0,0], distanceField:"distance", spherical:true, query:{$expr:{$eq:["$_id","$$var"]}}}})
2019-01-02T14:30:57.649-0500 E QUERY    [js] Error: command failed: {
	"ok" : 0,
	"errmsg" : "Can't parse filter / create query",
	"code" : 8,
	"codeName" : "UnknownError"
} : aggregate failed :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
doassert@src/mongo/shell/assert.js:18:14
_assertCommandWorked@src/mongo/shell/assert.js:534:17
assert.commandWorked@src/mongo/shell/assert.js:618:16
DB.prototype._runAggregate@src/mongo/shell/db.js:260:9
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1056:12
@(shell):1:1

Comment by David Storch [ 21/Dec/18 ]

While working on this further, brigitte.lamarche and I noticed that there is a similar bug for $text. The aggregation in the following repro script fails:

db.c.drop();
db.d.drop();
db.c.insert({});
db.d.insert({});
db.d.createIndex({a: "text"})
db.c.aggregate([{$lookup: {from: "d", as: "as", pipeline: [{$match: {$text: {$search: "foo"}}}]}}])

Unlike the $geoNear case, the $text bug affects 4.0 and 3.6. The fix for both $geoNear and $text will be the same, so we should consider backporting it.

Comment by David Storch [ 21/Dec/18 ]

This is a really good catch brigitte.lamarche. Nice work!

Generated at Thu Feb 08 04:49:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.