<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:14:10 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-7313] Add Nested Data (Associative Array) and Index Support</title>
                <link>https://jira.mongodb.org/browse/SERVER-7313</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Within a document structure, I find that a lot of times it is more intuitive and easier to handle/manipulate nested data stored as associative arrays than regular arrays. With associative arrays, I get to avoid (imho) the awkward positional operator and its sometimes slicing and manipulation limitations. Unfortunately for us, the support for querying/indexing against associative arrays is very limited which forces the designer to resort back to regular arrays or create an alternate (synced) structure for querying purposes only.&lt;/p&gt;


&lt;p&gt;For demonstration purposes, let&apos;s say we have a collection that tracks all classes and their students. One document for each class and a student can belong to many classes.&lt;/p&gt;


&lt;p&gt;If we use a regular array to store the student info, then we have...&lt;/p&gt;

&lt;p&gt;db.class.save({_id: &quot;class1&quot;, teacher:&quot;Chris&quot;, students: [&lt;br/&gt;
  {_id: &quot;asmith&quot;, grade: 97, rank: 1, stars:&lt;tt&gt;_id:&quot;reading&quot;,count:5}, {_id:&quot;math&quot;,count:8&lt;/tt&gt; },&lt;br/&gt;
  {_id: &quot;bsmith&quot;, grade: 90, rank: 2, stars:&lt;tt&gt;_id:&quot;reading&quot;,count:9}, {_id:&quot;math&quot;,count:3&lt;/tt&gt; }&lt;br/&gt;
]&lt;br/&gt;
});&lt;/p&gt;

&lt;p&gt;If we use an associative array to store the student info, then we have...&lt;/p&gt;

&lt;p&gt;db.classA.save({_id: &quot;class1&quot;, teacher:&quot;Chris&quot;, students: {&lt;br/&gt;
  &quot;asmith&quot; : {grade: 97, rank: 1, stars:{&quot;reading&quot;:&lt;/p&gt;
{count:5}
&lt;p&gt;, &quot;math&quot;:{count:8}} },&lt;br/&gt;
  &quot;bsmith&quot; : {grade: 90, rank: 2, stars:{&quot;reading&quot;:&lt;/p&gt;
{count:9}
&lt;p&gt;, &quot;math&quot;:{count:3}} }&lt;br/&gt;
}&lt;br/&gt;
});&lt;/p&gt;


&lt;p&gt;The following are some basic tasks that we would like to perform (against both collections). Criteria for performing the tasks is that they must be done in one command (atomic purposes) and any query must involve indexes only.&lt;/p&gt;



&lt;p&gt;-Pull only asmith&apos;s student info (not all students) from each of his classes:&lt;/p&gt;

&lt;p&gt;Regular: Not sure if that is possible?&lt;br/&gt;
Associative:  db.classA.find({&quot;students.asmith&quot;:{$exists:true}},&lt;/p&gt;
{&quot;students.asmith&quot;}
&lt;p&gt;); //see below about this weak index&lt;/p&gt;


&lt;p&gt;-Pull only asmith&apos;s student info along with the teacher&apos;s name of the class in one query:&lt;/p&gt;

&lt;p&gt;Regular: Not possible?&lt;br/&gt;
Associative: db.classA.find({&quot;students.asmith&quot;:{$exists:true}}, &lt;/p&gt;
{&quot;students.asmith&quot;,&quot;teacher&quot;}
&lt;p&gt;);&lt;/p&gt;


&lt;p&gt;-Update asmith&apos;s grade in class1:&lt;/p&gt;

&lt;p&gt;Regular: db.class.update({_id:&quot;class1&quot;, &quot;students._id&quot;:&quot;asmith&quot;}, {$set:{&quot;students.$.grade&quot;:96}});&lt;br/&gt;
Associative: db.classA.update({_id:&quot;class1&quot;}, {$set:{&quot;students.asmith.grade&quot;:96}});&lt;/p&gt;


&lt;p&gt;-Increment asmith reading star count (nested nested structure):&lt;/p&gt;

&lt;p&gt;Regular: Not possible (yet) - see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-831&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-831&lt;/a&gt;&lt;br/&gt;
Associative: db.classA.update({_id:&quot;class1&quot;}, {$inc:{&quot;students.asmith.stars.reading.count&quot;:1}});&lt;/p&gt;




&lt;p&gt;As the above examples demonstrate, it would be more straightforward to use an associative array.&lt;br/&gt;
However the indexing/querying support is limited so we cannot perform the following very basic task:&lt;/p&gt;


&lt;p&gt;-Search for students with a grade 90 or higher.&lt;/p&gt;

&lt;p&gt;Unlike the regular array structure, there is no way to search against grades using an index. I find this lack of support to be surprising given that arrays and associative arrays really only differ in keys; one is implicitly implied (sequential integer) while the other is explicitly set by the user. In theory, positional operator could work with both types of keys.&lt;/p&gt;


&lt;p&gt;Suggestion: Add index support at nested array level (regardless if is regular or associative):&lt;/p&gt;

&lt;p&gt;db.classA.ensureIndex(&lt;/p&gt;
{&quot;students.*.grade&quot;}
&lt;p&gt;);    //character indicates associative array; this could technically work with regular arrays as well&lt;br/&gt;
db.classA.find({&quot;students.*.grade&quot;:{$gte:90}}); //not only is query field name intuitive but it matches the same naming convention used in the index declaration&lt;br/&gt;
db.classA.update({&quot;students.*.grade&quot;:{$gte:90}}, {$inc:{&quot;students.$.grade&quot;:1}}); //positional operator example&lt;/p&gt;



&lt;p&gt;Consequences (Wins):&lt;/p&gt;

&lt;p&gt;-We are not forced thinking one way when structuring nested data or in some cases creating synced alternate data store to achieve our goals.&lt;/p&gt;

&lt;p&gt;The user should have the freedom to organize data and associate keys with it at the same time. In essence the data becomes true &quot;sub-documents&quot; where each data has its own unique identifier just like the main document record (_id).&lt;/p&gt;

&lt;p&gt;I cannot tell how many times I have come across users on the internet unaware of the index limitation and forced to restructure their data.&lt;/p&gt;


&lt;p&gt;-good &quot;$exists&quot; index performance against associative array keys is possible&lt;/p&gt;

&lt;p&gt;In the MongoDB documentation (Advanced Queries section), it states that &quot;$exists is not very efficient even with an index&quot;.&lt;br/&gt;
If indexes within associative arrays were supported, then the workaround to ensure good index performance to see if a user exists in a class is to store the username (key) inside the associative array values and index that field.&lt;/p&gt;

&lt;p&gt;E.g.&lt;br/&gt;
students: {&lt;br/&gt;
  &quot;asmith&quot; : {_id: &quot;asmith&quot;, grade: 97, rank: 1, ...},&lt;br/&gt;
  &quot;bsmith&quot; : {_id: &quot;bsmith&quot;, grade: 90, rank: 2, ...}&lt;br/&gt;
}&lt;/p&gt;


&lt;p&gt;db.classA.ensureIndex(&quot;students.*._id&quot;);&lt;br/&gt;
db.classA.query(&lt;/p&gt;
{&quot;students.*._id&quot;:&quot;asmith&quot;}
&lt;p&gt;);  //instead of using {&quot;students.asmith&quot;:{$exists:true}}&lt;/p&gt;


&lt;p&gt;Or better yet, maybe MongoDB can implicitly do that behind the scenes using some internal field when we index against the associative array.&lt;/p&gt;



&lt;p&gt;Other Readings:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://developer.bazaarvoice.com/mongodb-arrays-and-atomicity&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://developer.bazaarvoice.com/mongodb-arrays-and-atomicity&lt;/a&gt;  - Nice article on the benefits of using associative arrays (solved atomicity issues)&lt;/p&gt;</description>
                <environment></environment>
        <key id="52791">SERVER-7313</key>
            <summary>Add Nested Data (Associative Array) and Index Support</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="a_niceguy57">Steve K</reporter>
                        <labels>
                            <label>indexing</label>
                            <label>schema</label>
                    </labels>
                <created>Wed, 10 Oct 2012 15:30:59 +0000</created>
                <updated>Fri, 15 Feb 2013 15:06:53 +0000</updated>
                            <resolved>Thu, 17 Jan 2013 16:40:04 +0000</resolved>
                                                                    <component>Concurrency</component>
                    <component>Index Maintenance</component>
                    <component>Querying</component>
                    <component>Usability</component>
                                        <votes>6</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="242648" author="eliot" created="Thu, 17 Jan 2013 16:40:04 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-267&quot; title=&quot;Wildcard support in index/query/projection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-267&quot;&gt;SERVER-267&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 17 Jan 2013 16:40:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 4 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 4 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>a_niceguy57</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnlov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrjx8n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>29922</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht06cf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>