[SERVER-34064] $graphLookup not starting from collection Created: 22/Mar/18 Updated: 06/Dec/22 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 3.4.14, 3.6.3 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Staffan Eketorp | Assignee: | Backlog - Query Optimization |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query Optimization
|
| Participants: |
| Description |
|
Just got acquainted with the $graphLookup aggregation, but it seems a little clunky for my main use case. So I have a combined question and/or feature request: My use case is I want to fetch all descendants for a node in a tree structure. But I want them as plain documents. Thus I'm forced to something like:
It would be much more natural to me if the $graphLookup operator could operate without a starting collection, i.e. something like (I know this doesn't work now):
So, is this something that can be done today in some similar fashion. Otherwise, it'd be nice to have. |
| Comments |
| Comment by Staffan Eketorp [ 24/May/18 ] | |||||||||
|
Makes sense! | |||||||||
| Comment by Asya Kamsky [ 22/May/18 ] | |||||||||
|
Okay, I think understand now - I think in that case it would not be $graphLookup option but maybe either a $match option, or a new stage (like $graphMatch) or something like that...
| |||||||||
| Comment by Staffan Eketorp [ 16/May/18 ] | |||||||||
|
Right! With the addition: In the first working example, the first document will be replaced away which seems unnecessary. We could either skip it by passing another STARTID or have it added in the results, thereby not throwing away any data. But to the point - my main point has always been simple syntax and discoverability. $unwind and $replaceRoot are operators I've never used before (I think most people haven't) so it probably takes some time to come up with this pipeline. There are even blog posts on how to accomplish this. I think it should be fairly easy as it's a common case. | |||||||||
| Comment by Asya Kamsky [ 16/May/18 ] | |||||||||
|
I'm afraid I don't see how this is different than the original pipeline you use - is it that you want to be able to write it as a single stage for simplicity?
| |||||||||
| Comment by Staffan Eketorp [ 05/May/18 ] | |||||||||
|
Ok, I understand if it doesn't make sense architecture wise. But do you acknowledge my use case? For me it's a MUCH more common use case to do plain document matching via a graph pattern, than lookup extra fields for existing documents. How about the following then? The $graphMatch operator would not need a "from" as it has the underlying collection. It would not add fields to existing documents but only recursively match "main documents" - in that way behaving more like $match. I guess the $graphLookup could be used for this as well, with an option like "matchStart": true, or similar.
| |||||||||
| Comment by Asya Kamsky [ 04/May/18 ] | |||||||||
|
The way aggregation works is as a pipeline. Initial collection provides input documents to process in the pipeline. For each document flowing through the pipeline something needs to get done in each stage. The suggested syntax would have zero documents entering the pipeline so ... something else has to create that document - that's what {match: {_id: 'STARTID'}} does, it limits the incoming documents from source collection to ones that match the condition. I'm not sure I see how the alternative syntax makes sense in the current aggregation flow. | |||||||||
| Comment by Staffan Eketorp [ 06/Apr/18 ] | |||||||||
|
Thanks for the reply. Not sure if I understood you correctly. I don't suggest 2 different syntax (or parameter sets) for the $graphLookup stage, but only to be able to run $graphLookup without an underlying collection. If you argue that supporting something like that would introduce more confusion than clarity I would disagree. Speaking from own experience, today's operator set was indeed confusing and I had to spend significant time trying to get it to work, finally finding an answer in a blog post. | |||||||||
| Comment by Asya Kamsky [ 30/Mar/18 ] | |||||||||
|
staeke note that the "$unwind" stage gets absorbed into $graphLookup during execution (you can see that by running explain) so only two stages are executed. It seems like the $replaceRoot stage is doing exactly what you need it for - many other use cases would want to preserve top level fields from the original document. I'm not sure that having two different syntax options in the same $graphLookup stage wouldn't be more confusing for them. |