[SERVER-51121] $graphqLookup with dynamic value evaluation in restrictSearchWithMatch expression Created: 24/Sep/20  Updated: 15/Jul/21  Resolved: 15/Jul/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Hitesh Baldaniya Assignee: Edwin Zhou
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Here is the example 

hdbrs:PRIMARY> db.models.find()
{ "_id" : ObjectId("5f6c72cec48056980f22d3d0"), "id" : "a1", "refs" : [ { "id" : "b1" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "e1" } ] }
{ "_id" : ObjectId("5f6c72e3c48056980f22d3d1"), "id" : "c1", "refs" : [ { "id" : "c11" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "c12" }, { "id" : "c13" } ] }
{ "_id" : ObjectId("5f6c72f2c48056980f22d3d2"), "id" : "b1", "refs" : [ { "id" : "b11" }, { "id" : "b12" }, { "id" : "b13" } ] }
{ "_id" : ObjectId("5f6c72ffc48056980f22d3d3"), "id" : "d1", "refs" : [ { "id" : "d11" }, { "id" : "d12" }, { "id" : "d13" } ] }
{ "_id" : ObjectId("5f6c78d7c48056980f22d3d4"), "id" : "c11", "refs" : [ { "id" : "c21" }, { "id" : "c22" } ] }

In the above example, when we search for the "c12" and "d13" id and all of it's parents using aggregation.

hdbrs:PRIMARY> db.models.aggregate([{$match: {"refs.id": {$in: searchIds}}}, {$graphLookup: {from: "models", startWith: searchIds, connectFromField: "id", connectToField: "refs.id", as: "Tree", maxDepth: 2, depthField: "depth" }}, {$project: {"Tree.id": 1, "Tree.depth": 1, _id: 0}}], {explain: false})
 
// Results 
 
{ "Tree" : [ { "id" : "c1", "depth" : NumberLong(1) }, { "id" : "a1", "depth" : NumberLong(1) }, { "id" : "c11", "depth" : NumberLong(0) }, { "id" : "d1", "depth" : NumberLong(0) } ] }
{ "Tree" : [ { "id" : "a1", "depth" : NumberLong(1) }, { "id" : "c1", "depth" : NumberLong(1) }, { "id" : "d1", "depth" : NumberLong(0) }, { "id" : "c11", "depth" : NumberLong(0) } ] }

As per the results you can see both documents contain the same document ids. If we allow dynamic search on the value "Tree.id" where the condition

{restrictSearchWithMatch: {$nin: "Tree.id"}}

will stop the repeated documents from getting again into the results set and instead of the different documents from the result set can we get output something like a single document as result set.



 Comments   
Comment by Edwin Zhou [ 15/Jul/21 ]

Hi hitesh.baldaniya@contentstack.com,

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Best,
Edwin

Comment by Edwin Zhou [ 22/Jun/21 ]

Hi hitesh.baldaniya@contentstack.com,

We still need additional information to diagnose the problem. If this is still an issue for you, would you please further specify what was unexpected in your given result set, as well as example documents that you expect to see when running the aggregation?

Best,
Edwin

Comment by Edwin Zhou [ 02/Jun/21 ]

Hi hitesh.baldaniya@contentstack.com,

I apologize for the delay in our response; we're extremely grateful for your patience.

I'd first like to stress that the server currently supports checking objects to see if they have already been evaluated. We would otherwise end up in an infinite loop.

In the example you provided, could you further specify what was unexpected in your given result set, as well as example documents that you expect to see when running the aggregation?

Best,
Edwin

Comment by Hitesh Baldaniya [ 29/Sep/20 ]

Hi @Jonathan,

I am actually making a kind of feature request, "$graphLookup" is working perfectly fine . 

What I am looking for is kind of like when we iterate through the objects we can check if they are already evaluated or not.

Actual graph like evaluation where we do not visit the already visited node in the Graph. While in GraphLookup what happens is something like for each node we identify the all of it's parents nodes but in my case I am looking for global graph where how each node is connected with other and in that if node is already visited we do not have to identify the their linked nodes again.

Let me know if you still have any questions.

Comment by Jonathan Streets (Inactive) [ 28/Sep/20 ]

hi hitesh.baldaniya@contentstack.com,
Thank-you for the information, I can reproduce the result you are seeing. however when i remove the projection i see that the 2 documents have different ids:

db.models.aggregate([{$match: {"refs.id": {$in: searchIds}}}, {$graphLookup: {from: "models", startWith: searchIds, connectFromField: "id", connectToField: "refs.id", as: "Tree", maxDepth: 2, depthField: "depth" }}], {explain: false})
{ "_id" : ObjectId("5f6c72e3c48056980f22d3d1"), "id" : "c1", "refs" : [ { "id" : "c11" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "c12" }, { "id" : "c13" } ], "Tree" : [ { "_id" : ObjectId("5f6c72e3c48056980f22d3d1"), "id" : "c1", "refs" : [ { "id" : "c11" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "c12" }, { "id" : "c13" } ], "depth" : NumberLong(0) }, { "_id" : ObjectId("5f6c72cec48056980f22d3d0"), "id" : "a1", "refs" : [ { "id" : "b1" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "e1" } ], "depth" : NumberLong(1) }, { "_id" : ObjectId("5f6c72ffc48056980f22d3d3"), "id" : "d1", "refs" : [ { "id" : "d11" }, { "id" : "d12" }, { "id" : "d13" } ], "depth" : NumberLong(0) } ] }
{ "_id" : ObjectId("5f6c72ffc48056980f22d3d3"), "id" : "d1", "refs" : [ { "id" : "d11" }, { "id" : "d12" }, { "id" : "d13" } ], "Tree" : [ { "_id" : ObjectId("5f6c72e3c48056980f22d3d1"), "id" : "c1", "refs" : [ { "id" : "c11" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "c12" }, { "id" : "c13" } ], "depth" : NumberLong(0) }, { "_id" : ObjectId("5f6c72cec48056980f22d3d0"), "id" : "a1", "refs" : [ { "id" : "b1" }, { "id" : "c1" }, { "id" : "d1" }, { "id" : "e1" } ], "depth" : NumberLong(1) }, { "_id" : ObjectId("5f6c72ffc48056980f22d3d3"), "id" : "d1", "refs" : [ { "id" : "d11" }, { "id" : "d12" }, { "id" : "d13" } ], "depth" : NumberLong(0) } ] }

Can you tell me more about what you would like to see ?
thanks
Jon

 

Generated at Thu Feb 08 05:24:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.