<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:23:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-72874] PoC - Coarse Grained Streaming Group</title>
                <link>https://jira.mongodb.org/browse/SERVER-72874</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Follow up to streaming group, this has the potential to remove sort if all buckets are sorted(or from compressed format). Could be useful to test the solution first before committing to make the change.&#160;&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Coarse Grained Streaming Group
	&lt;ul&gt;
		&lt;li&gt;Observation: locally sorted runs, globally &#8220;coarsely&#8221; sorted. Underscored numbers are min time for each bucket.
		&lt;ul&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,14,&lt;ins&gt;13&lt;/ins&gt;,14,15,16,17,&lt;ins&gt;16&lt;/ins&gt;,17,18.19.20&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
		&lt;li&gt;Leading active groups can be closed at &#8220;sorted run boundary&#8221;, &lt;span class=&quot;error&quot;&gt;&amp;#91;...&amp;#93;&lt;/span&gt; represent groups, closed groups have strike-through markings.
		&lt;ul&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,14&#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &lt;span class=&quot;error&quot;&gt;&amp;#91;10-12&amp;#93;&lt;/span&gt;,&lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;&lt;/li&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,14,&lt;ins&gt;13&lt;/ins&gt; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &lt;del&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;10-12&amp;#93;&lt;/span&gt;&lt;/del&gt;&lt;del&gt;,&lt;/del&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;&lt;/li&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,14,&lt;ins&gt;13&lt;/ins&gt;,14,15,16,17 &#160; &#160; &#160; &#160; &lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;,&lt;span class=&quot;error&quot;&gt;&amp;#91;16-18&amp;#93;&lt;/span&gt;&lt;/li&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,14,&lt;ins&gt;13&lt;/ins&gt;,14,15,16,17,&lt;ins&gt;16&lt;/ins&gt;&#160; &#160; &lt;del&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;&lt;/del&gt;,&lt;span class=&quot;error&quot;&gt;&amp;#91;16-18&amp;#93;&lt;/span&gt;&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
		&lt;li&gt;Current State(as of PM-3050): Current Streaming group requires global sort (optimized as &#8220;bounded&#8221; sort needing a local heap to sort the run up to the next &quot;bound&quot;).
		&lt;ul&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,&lt;ins&gt;13&lt;/ins&gt;,14,14,15,16,&lt;ins&gt;16&lt;/ins&gt;,17,17,18.19.20&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
		&lt;li&gt;Current State: but can close the leading group earlier (less memory usage) once input is sorted (group bounds to be compared to the next value to determine whether to output an active group)
		&lt;ul&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12 &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &lt;span class=&quot;error&quot;&gt;&amp;#91;10-12&amp;#93;&lt;/span&gt;&lt;/li&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13&#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &lt;del&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;10-12&amp;#93;&lt;/span&gt;&lt;/del&gt;,&lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;&lt;/li&gt;
			&lt;li&gt;&lt;ins&gt;10&lt;/ins&gt;,11,12,13,&lt;ins&gt;13&lt;/ins&gt;,14,14,15,16 &#160; &#160; &#160; &#160; &#160; &lt;del&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;13-15&amp;#93;&lt;/span&gt;&lt;/del&gt;,&lt;span class=&quot;error&quot;&gt;&amp;#91;16-18&amp;#93;&lt;/span&gt;&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
		&lt;li&gt;How early is a function of the group size and amount of overlap&lt;/li&gt;
		&lt;li&gt;This proposal essentially combines run boundary detection with grouping (ie compare group bundary against run boundary to determine whether to stream the results from an active group or not)
		&lt;ul&gt;
			&lt;li&gt;We believe this would improve the query performance due to one fewer step in the pipeline as the sort could be eliminated.&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
		&lt;li&gt;How to detect &#8220;sorted run boundary&#8221;: Extra info to indicate bucket start/end is provided in the stream.
		&lt;ul&gt;
			&lt;li&gt;Borrow from the &#8220;&lt;a href=&quot;https://docs.google.com/document/d/1d1artozDJUU1WmwHfcViV2BPIaDjns-xcylvPDuHiiI/edit#heading=h.ic8ludepnspp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;bounded sort&lt;/a&gt;&#8221; implementation:
			&lt;ul&gt;
				&lt;li&gt;&#8220; It also can see the control.min.time value of the containing bucket. Whenever it sees the control.min.time change, it knows that no subsequent input will have a lower value, and so anything below this value can be returned.&#8221;&lt;/li&gt;
			&lt;/ul&gt;
			&lt;/li&gt;
			&lt;li&gt;Info made available to sort via &lt;a href=&quot;https://github.com/10gen/mongo/blob/2cb27e77bffdf3ce9ee1715904a3870f764e5ade/src/mongo/db/pipeline/document_source_sort.cpp#L61&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;BoundMakerMin/Max&lt;/a&gt;&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2234129">SERVER-72874</key>
            <summary>PoC - Coarse Grained Streaming Group</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="ivan.fefer@mongodb.com">Ivan Fefer</assignee>
                                    <reporter username="rushan.chen@mongodb.com">Rushan Chen</reporter>
                        <labels>
                    </labels>
                <created>Mon, 16 Jan 2023 15:20:01 +0000</created>
                <updated>Mon, 17 Apr 2023 12:20:38 +0000</updated>
                            <resolved>Mon, 17 Apr 2023 12:20:38 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="5351966" author="JIRAUSER1270969" created="Mon, 17 Apr 2023 12:20:05 +0000"  >&lt;p&gt;Prototype implementation showed that the support of &quot;close some, but not all groups&quot; in DocumentSourceGroup will generally lead to performance regression of ~10%. For timeseries case we need to support a ordered set of all current groups by time. This outweighs gains for reduced RAM consumption.&#160;&lt;/p&gt;</comment>
                            <comment id="5251338" author="JIRAUSER1270969" created="Mon, 6 Mar 2023 11:21:21 +0000"  >&lt;p&gt;Sent to backlog until I have some time&lt;/p&gt;</comment>
                            <comment id="5213756" author="JIRAUSER1270969" created="Mon, 20 Feb 2023 12:33:22 +0000"  >&lt;p&gt;Currently pausing my work on this due to priority&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 20 Feb 2023 12:33:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        42 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ivan.fefer@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            42 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>ivan.fefer@mongodb.com</customfieldvalue>
            <customfieldvalue>rushan.chen@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1qs3r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i19avs:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5930">QE 2023-02-06</customfieldvalue>
    <customfieldvalue id="5932">QE 2023-02-20</customfieldvalue>
    <customfieldvalue id="6990">QE 2023-03-06</customfieldvalue>
    <customfieldvalue id="6992">QE 2023-03-20</customfieldvalue>
    <customfieldvalue id="6994">QE 2023-04-03</customfieldvalue>
    <customfieldvalue id="6996">QE 2023-04-17</customfieldvalue>
    <customfieldvalue id="6998">QE 2023-05-01</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1qe93:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>