<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:20:03 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-29161] Ability to access previous document in $group aggregation</title>
                <link>https://jira.mongodb.org/browse/SERVER-29161</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When performing an aggregation query in mongodb, it would sometimes be useful to reference the previous document. As we have the `$$ROOT` variable, perhaps we could have `$$PREVIOUS` or some other meaningful name that let&apos;s us do document to document calculations.&lt;/p&gt;

&lt;p&gt;A possible use case would be calculating time passed in between documents more easily, and calculating deltas between documents. &lt;/p&gt;
</description>
                <environment></environment>
        <key id="383037">SERVER-29161</key>
            <summary>Ability to access previous document in $group aggregation</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="arg20">Gabriel Zimmermann</reporter>
                        <labels>
                            <label>expression</label>
                    </labels>
                <created>Fri, 12 May 2017 16:38:19 +0000</created>
                <updated>Tue, 6 Dec 2022 04:01:12 +0000</updated>
                            <resolved>Tue, 1 Jun 2021 18:49:51 +0000</resolved>
                                    <version>3.5.6</version>
                                                    <component>Aggregation Framework</component>
                                        <votes>8</votes>
                                    <watches>19</watches>
                                                                                                                <comments>
                            <comment id="3852088" author="JIRAUSER1258718" created="Tue, 1 Jun 2021 18:49:51 +0000"  >&lt;p&gt;In 5.0, you will be able to use $shift and $setWindowFields to achieve this.&lt;/p&gt;</comment>
                            <comment id="2658897" author="tyeager@real.com" created="Fri, 20 Dec 2019 22:26:24 +0000"  >&lt;p&gt;We want to coalesce image recognition events that have a start and end time.&#160; If the gap between the start time of the $$CURRENT event and the end of the $$PREVIOUS event is less than some duration, we would group the documents into a new event.&#160; &#160;&lt;/p&gt;</comment>
                            <comment id="2584492" author="brett.gray" created="Thu, 5 Dec 2019 00:15:18 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt; I have a customer that could also use the suggested &lt;tt&gt;$$PREVIOUS&lt;/tt&gt; to assist in calculate\ing variance of products created compared to the previous week.&lt;/p&gt;</comment>
                            <comment id="1777078" author="asya" created="Wed, 17 Jan 2018 20:10:13 +0000"  >&lt;p&gt;It&apos;s conceivable that this would be a useful expression in any stage (i.e. $project/$addFields).&lt;/p&gt;</comment>
                            <comment id="1614068" author="asya" created="Wed, 5 Jul 2017 14:28:23 +0000"  >&lt;blockquote&gt;&lt;p&gt;when I&apos;m processing the second document for that `group`, `$$PREVIOUS` will be a reference to the first document, and so on. so that I can do something like &quot;$$PREVIOUS.temperature - $$ROOT.temperature&quot;.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Okay, I think I understand the example.  I do think the linked ticket, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-29339&quot; title=&quot;allow using $reduce expression as accumulator in $group&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-29339&quot;&gt;SERVER-29339&lt;/a&gt; would allow something similar, by allowing saving/accumulating previous value(s).   We will need to figure out which syntax would allow for more readable (and/or more performant) implementation.&lt;/p&gt;</comment>
                            <comment id="1613542" author="arg20" created="Tue, 4 Jul 2017 14:22:57 +0000"  >&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;My question was due to having some solutions similar to the one Charlie Swanson linked to. In general, such computations are done for a particular time period, right? So pushing all deltas for each &quot;device&quot; to an array should be pretty doable. Though I&apos;m guessing you don&apos;t mean deltas but rather storing only values that have changed since previous reading?&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Yeah, in some cases we store values that have only changed since the previous reading, in other case we are particularly interested in storing deltas directly. However, I think with the rise of IoT applications (and my company is particularly invested in this), the need to store sensor data or data from devices that produce a really large volume of information is growing, and pushing deltas for each device might become expensive/convoluted. Our aggregation queries are complex enough at the moment unfortunately and they take their fair amount of seconds to execute.&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;The issue with providing &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&quot;$$PREVIOUS&quot;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; is that &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;if&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; the previous document was somehow transformed during the processing, so should &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;this&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; be the document before it was &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&quot;processed&quot;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; or after?&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Please excuse my ignorance here, I do not know how the `$group` stage works under the hood so this request is honestly coming from my lack of knowledge. What I meant was that I want `$$PREVIOUS` (or more suitable name), to point to the previous `$ROOT` processed for that group. So that, if my group is `_id: &quot;$deviceId&quot;`, when I&apos;m processing the second document for that `group`, `$$PREVIOUS` will be a reference to the first document, and so on. so that I can do something like &quot;$$PREVIOUS.temperature - $$ROOT.temperature&quot;.&lt;/p&gt;

&lt;p&gt;Does that make sense? Thanks&lt;/p&gt;</comment>
                            <comment id="1573679" author="asya" created="Wed, 17 May 2017 18:19:19 +0000"  >&lt;p&gt;My question was due to having some solutions similar to the one &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt; linked to.  In general, such computations are done for a particular time period, right?  So pushing all deltas for each &quot;device&quot; to an array should be pretty doable. Though I&apos;m guessing you don&apos;t mean deltas but rather storing only values that have changed since previous reading?&lt;/p&gt;

&lt;p&gt;The issue with providing &quot;$$PREVIOUS&quot; is that if the previous document was somehow transformed during the processing, so should this be the document before it was &quot;processed&quot; or after?  Comparing elements in the array is simple enough (another example is &lt;a href=&quot;http://www.kamsky.org/stupid-tricks-with-mongodb/how-to-do-intra-array-comparisons&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; but it&apos;s less clear to me that providing $$PREVIOUS would make the resultant pipeline easier to understand.&lt;/p&gt;

&lt;p&gt;We will discuss this request during our next planning cycle, thanks for the suggestion!&lt;/p&gt;</comment>
                            <comment id="1573352" author="arg20" created="Wed, 17 May 2017 14:50:43 +0000"  >&lt;p&gt;Yeah, in my particular case, the size of our collection makes it almost impossible for us to create an array, unwind etc. That&apos;s clever though. I think the big picture is that when you are grouping, you often need a bit of context to process a given document, having access to the previous value makes it easier to perform conditional computations.&lt;/p&gt;</comment>
                            <comment id="1573319" author="charlie.swanson" created="Wed, 17 May 2017 14:29:07 +0000"  >&lt;p&gt;I think &lt;a href=&quot;http://stackoverflow.com/questions/43560777/mongodb-aggregation-query-to-subtract-and-grouping-of-cumulative-value/43594156#43594156&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this stack overflow question&lt;/a&gt; is related, I had to do some crazy stuff to reference the previous values.&lt;/p&gt;</comment>
                            <comment id="1572906" author="arg20" created="Wed, 17 May 2017 01:15:22 +0000"  >&lt;p&gt;Asya. One of the use cases is for IoT projects my company is developing for a big client. Most of this data is sensor measurements across time. At some point the volume of the data became too large and we moved to storing only changes in the measurements. So instead of storing things like temperature value every 5 seconds, we store only deltas in the temperature. &lt;/p&gt;

&lt;p&gt;Since we are dealing with changes, sometimes we want to know calculations based on these deltas. So in each document I&apos;m forced to store the current value and the previous value so I can compare them. Sometimes I need to know if the temperature increased with respect to the last one, and I can&apos;t determine this at the moment in which I store this or I&apos;d store it pre-calculated already. Other times I need to perform conditional logic based on whether a previous distance reading was greater/less than the current reading in a $group operation.&lt;/p&gt;

&lt;p&gt;It boils down to the fact that when you&apos;re storing deltas to some piece of information, often to reconstruct it, when you $group you need to perform delta-to-delta calculations. If I had access to the `previous` document, it would be trivial to do this, but for now each document stores its data, plus a bulk of data from the previous one.&lt;/p&gt;

&lt;p&gt;I think in general this would enable the aggregation framework to achieve a new level of complexity in the algorithms you can write, specially when the volume of the data is so large that it becomes convenient to store only changes to the data.&lt;/p&gt;</comment>
                            <comment id="1572328" author="asya" created="Tue, 16 May 2017 15:16:59 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=arg20&quot; class=&quot;user-hover&quot; rel=&quot;arg20&quot;&gt;arg20&lt;/a&gt; can you give a brief description of your use case?  What calculation/processing you are currently trying to do, etc?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="1610318">SERVER-54243</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="26052">SERVER-4437</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="32139">SERVER-5192</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="386266">SERVER-29339</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 16 May 2017 15:15:56 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 36 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1834</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 36 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>brett.gray@mongodb.com</customfieldvalue>
            <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>arg20</customfieldvalue>
            <customfieldvalue>joseph.kanaan@mongodb.com</customfieldvalue>
            <customfieldvalue>tyeager@real.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht7cav:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr36av:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsb2rr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>