[SERVER-58411] Deep find nested objects in return documents. Created: 10/Jul/21 Updated: 06/May/22 Resolved: 29/Mar/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Gerrie van Wyk | Assignee: | Chris Harris |
| Resolution: | Won't Do | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Sprint: | QE 2021-08-09, QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04, QE 2021-10-18, QE 2021-11-01, QE 2021-11-15, QE 2021-11-29, QE 2021-12-13, QE 2021-12-27, QE 2022-01-10, QE 2022-01-24 | ||||||||
| Participants: | |||||||||
| Description |
|
Not sure how feasible this is, but working with relational data in MongoDB is always a pain since finding nested documents requires writing an aggregation using $lookup, $unwind, etc. Just thought of a convenient implementation to solve this. If only ObjectIds included a collection id, then you could call a "deep find" method which fetches all nested objects where it encounters an ObjectId in the return documents. Other considerations here would be infinite loop protection and a max depth feature. |
| Comments |
| Comment by Chris Harris [ 29/Mar/22 ] | ||||||
|
Hi gvw@lowdefy.com, Thanks again for filing this request. As outlined by Asya above - your desired functionality is mostly achievable today by using DBRefs. Alternatively you could consider using Views to store the aggregation pipeline in the database itself, which can simplify how the client application(s) expresses read requests. Based on this, we do not currently have any plans to implement this functionality and will therefore proceed to close this ticket. It is also worth noting that, as with any database system, appropriately modeling your data is important for long-term success. We have a variety of resources available on the topic, but please do reach out to the community on the forums if you have questions or would like more information in this regard. Best, | ||||||
| Comment by Asya Kamsky [ 20/Jul/21 ] | ||||||
|
gvw@lowdefy.com I'm not sure I understand the original problem. You said: > finding nested documents requires writing an aggregation using $lookup, $unwind Generally finding matching nested documents does not require $unwind neither does returning matching subdocuments. $lookup is not really related to finding documents but rather joining together two collections, but that also shouldn't require extra stages (obviously it requires aggregation since it's a stage in agg). With the example you provide it looks like you are using the term "nested documents" to mean "related documents in a different collection(s)" rather than what we usually mean (which is having an array of documents inside the document)... And the mechanism you are describing looks similar to DBRef which is meant to be used by the driver(s) or other external frameworks to resolve such relationships (and has some advantages but also disadvantages). Note that we cannot resolve such references in the server because we check collection access permissions at parse time and here collection is specified inside the data dynamically. SERVER-22497 asks for ability to specify it as an expression (including a dynamic field in the document) but our AUTH model is the reason we cannot implement it that way. | ||||||
| Comment by Gerrie Van wyk [ 16/Jul/21 ] | ||||||
|
Hi @Eric Thanks for looking at this. Perhaps let me try and explain what I feel would be an "nice to have" with an example: Painpoint Working with relational data in mongo is challenging simply because in order to fetch the document and nested documents, one needs to run an aggregation and use $lookup, handle $unwind where required. Solution As a dev, it would be great to have:
From a user perspective this would make it really easy to work with relational data, instead of writing aggregations to deal with all this. Currently, I suspect the above would not be possible because ObjectId does not contain any information for which collection it is created / stored in so the find method has no idea where to look to the relational document. Hence you need to provide the collection and lined field in the $lookup stage for the aggregation. As an future idea, if ObjectIDs contained a short collection id, then the mongo server would know where to go and find the any nested relational document - thus a deepFind method can be added to resolve the nested documents linked using the improved ObjectIDs. Obviously I have no idea how complex such an implementation would be if even possible, however, from an end user / dev perspective it would make working with relational data in MongoDB sooo nice.
| ||||||
| Comment by Eric Sedor [ 15/Jul/21 ] | ||||||
|
Thanks gvw@lowdefy.com. I read this as a request for some form of dynamic, recursive $lookup with the pain point being that it's a challenge to keep complex document structures and complex aggregation pipelines in agreement. I'll pass this on to an appropriate team to consider. But please feel free to clarify or elaborate on your use-case. Eric |