<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:44:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-58411] Deep find nested objects in return documents.</title>
                <link>https://jira.mongodb.org/browse/SERVER-58411</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Not sure how feasible this is, but working with relational data in MongoDB is always a pain since finding nested documents requires writing an aggregation using $lookup, $unwind, etc.&#160;&lt;/p&gt;

&lt;p&gt;Just thought of a convenient implementation to solve this. If only ObjectIds included a collection id, then you could call a &quot;deep find&quot; method which fetches all nested objects where it encounters an ObjectId in the return documents.&#160;&lt;/p&gt;

&lt;p&gt;Other considerations here would be infinite loop protection and a max depth feature.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1814374">SERVER-58411</key>
            <summary>Deep find nested objects in return documents.</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="christopher.harris@mongodb.com">Chris Harris</assignee>
                                    <reporter username="gvw@lowdefy.com">Gerrie van Wyk</reporter>
                        <labels>
                    </labels>
                <created>Sat, 10 Jul 2021 19:09:50 +0000</created>
                <updated>Fri, 6 May 2022 17:57:36 +0000</updated>
                            <resolved>Tue, 29 Mar 2022 14:53:30 +0000</resolved>
                                                                                        <votes>2</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="4441881" author="christopher.harris" created="Tue, 29 Mar 2022 14:53:08 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gvw%40lowdefy.com&quot; class=&quot;user-hover&quot; rel=&quot;gvw@lowdefy.com&quot;&gt;gvw@lowdefy.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks again for filing this request.&lt;/p&gt;

&lt;p&gt;As outlined by Asya above - your desired functionality is mostly achievable today by using DBRefs.&#160; Alternatively you could consider using &lt;a href=&quot;https://www.mongodb.com/docs/manual/core/views&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Views&lt;/a&gt;&#160;to store the aggregation pipeline in the database itself, which can simplify how the client application(s) expresses read requests.&#160; Based on this, we do not currently have any plans to implement this functionality and will therefore proceed to close this ticket.&lt;/p&gt;

&lt;p&gt;It is also worth noting that, as with any database system, appropriately modeling your data is important for long-term success.&#160; We have a variety of resources available on the topic, but please do reach out to the community &lt;a href=&quot;https://www.mongodb.com/community/forums/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;on the forums&lt;/a&gt;&#160;if you have questions or would like more information in this regard.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Chris&lt;/p&gt;</comment>
                            <comment id="3952790" author="asya" created="Tue, 20 Jul 2021 16:06:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gvw%40lowdefy.com&quot; class=&quot;user-hover&quot; rel=&quot;gvw@lowdefy.com&quot;&gt;gvw@lowdefy.com&lt;/a&gt; I&apos;m not sure I understand the original problem.  You said:&lt;/p&gt;

&lt;p&gt;&amp;gt; finding nested documents requires writing an aggregation using $lookup, $unwind&lt;/p&gt;

&lt;p&gt;Generally finding matching nested documents does not require $unwind neither does returning matching subdocuments.&lt;/p&gt;

&lt;p&gt;$lookup is not really related to finding documents but rather joining together two collections, but that also shouldn&apos;t require extra stages (obviously it requires aggregation since it&apos;s a stage in agg).&lt;/p&gt;

&lt;p&gt;With the example you provide it looks like you are using the term &quot;nested documents&quot; to mean &quot;related documents in a different collection(s)&quot; rather than what we usually mean (which is having an array of documents inside the document)...   And the mechanism you are describing looks similar to &lt;a href=&quot;https://docs.mongodb.com/manual/reference/database-references/#dbrefs&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;DBRef&lt;/a&gt; which is meant to be used by the driver(s) or other external frameworks to resolve such relationships (and has some advantages but also disadvantages).&lt;/p&gt;

&lt;p&gt;Note that we cannot resolve such references in the server because we check collection access permissions at parse time and here collection is specified inside the data dynamically.   &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22497&quot; title=&quot;Allow from field of $lookup and $graphLookup to be an expression&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22497&quot;&gt;SERVER-22497&lt;/a&gt; asks for ability to specify it as an expression (including a dynamic field in the document) but our AUTH model is the reason we cannot implement it that way.&lt;/p&gt;

</comment>
                            <comment id="3946441" author="gerrie@mrmtech.co" created="Fri, 16 Jul 2021 09:45:03 +0000"  >&lt;p&gt;Hi @Eric&lt;/p&gt;

&lt;p&gt;Thanks for looking at this. Perhaps let me try and explain what I feel would be an &quot;nice to have&quot; with an example:&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Painpoint&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Working with relational data in mongo is challenging simply because in order to fetch the document and nested documents, one needs to run an aggregation and use $lookup, handle $unwind where required.&#160;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Solution&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;As a dev, it would be great to have:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;const&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; res_A = collection_A.insertOne({ a: &lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;const&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; res_B = collection_B.insertOne({ b: &lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, a: res_A.insertedId });&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;const&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; res = collection_B.deepFind({ _id: res_B._id });&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #008200; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// where res is:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #008200; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// res = { _id: ObjectID(&apos;abc&apos;), b: 2, a: { _id: ObjectID(&apos;xyz&apos;), a: 1} }&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;From a user perspective this would make it really easy to work with relational data, instead of writing aggregations to deal with all this.&lt;/p&gt;

&lt;p&gt;Currently, I suspect the above would not be possible because ObjectId does not contain any information for which collection it is created / stored in so the find method has no idea where to look to the relational document. Hence you need to provide the collection and lined field in the $lookup stage for the aggregation.&lt;/p&gt;

&lt;p&gt;As an future idea, if ObjectIDs contained a short collection id, then the mongo server would know where to go and find the any nested relational document - thus a deepFind method can be added to resolve the nested documents linked using the improved ObjectIDs.&#160;&lt;/p&gt;

&lt;p&gt;Obviously I have no idea how complex such an implementation would be if even possible, however, from an end user / dev perspective it would make working with relational data in MongoDB sooo nice.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3944447" author="eric.sedor" created="Thu, 15 Jul 2021 19:01:29 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gvw%40lowdefy.com&quot; class=&quot;user-hover&quot; rel=&quot;gvw@lowdefy.com&quot;&gt;gvw@lowdefy.com&lt;/a&gt;. I read this as a request for some form of dynamic, recursive $lookup with the pain point being that it&apos;s a challenge to keep complex document structures and complex aggregation pipelines in agreement. I&apos;ll pass this on to an appropriate team to consider. But please feel free to clarify or elaborate on your use-case.&lt;/p&gt;

&lt;p&gt;Eric&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="263524">SERVER-22497</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>13.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 15 Jul 2021 19:01:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 45 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>joe.sack@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 45 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>christopher.harris@mongodb.com</customfieldvalue>
            <customfieldvalue>eric.sedor@mongodb.com</customfieldvalue>
            <customfieldvalue>gerrie@mrmtech.co</customfieldvalue>
            <customfieldvalue>gvw@lowdefy.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzrhyv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hzbstj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4713">QE 2021-08-09</customfieldvalue>
    <customfieldvalue id="4715">QE 2021-08-23</customfieldvalue>
    <customfieldvalue id="4717">QE 2021-09-06</customfieldvalue>
    <customfieldvalue id="4719">QE 2021-09-20</customfieldvalue>
    <customfieldvalue id="5265">QE 2021-10-04</customfieldvalue>
    <customfieldvalue id="5267">QE 2021-10-18</customfieldvalue>
    <customfieldvalue id="5269">QE 2021-11-01</customfieldvalue>
    <customfieldvalue id="5271">QE 2021-11-15</customfieldvalue>
    <customfieldvalue id="5273">QE 2021-11-29</customfieldvalue>
    <customfieldvalue id="5275">QE 2021-12-13</customfieldvalue>
    <customfieldvalue id="5277">QE 2021-12-27</customfieldvalue>
    <customfieldvalue id="5279">QE 2022-01-10</customfieldvalue>
    <customfieldvalue id="5810">QE 2022-01-24</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[eric.sedor@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzr47z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>