<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:02:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DOCS-11102] Docs for SERVER-24981: $project-$limit optimization has bad repercussion on pipeline splitting</title>
                <link>https://jira.mongodb.org/browse/DOCS-11102</link>
                <project id="10380" key="DOCS">Documentation</project>
                    <description>&lt;h1&gt;&lt;a name=&quot;DocumentationRequestSummary%3A&quot;&gt;&lt;/a&gt;Documentation Request Summary: &lt;/h1&gt;

&lt;h2&gt;&lt;a name=&quot;Description&quot;&gt;&lt;/a&gt;Description&lt;/h2&gt;

&lt;p&gt;$sort now searches the entire pipeline for a $limit and if found, coalesces the $limit into itself. If there is a stage in between the $sort and $limit that changes the number of documents in the pipeline (i.e. $group, $unwind, etc.), the $sort aborts its search for a $limit. An exception to this rule is the case where one or multiple $skip stages are in between a $sort and $limit. In this case, $sort will still coalesce the $limit, but the $limit value increases by the total of the amounts of all of the $skip stages in between. This means that neither $project or $skip swap with $limit anymore if $sort is not present.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;Scopeofchanges%28filesthatneedworkandhowmuch%29&quot;&gt;&lt;/a&gt;Scope of changes (files that need work and how much)&lt;/h2&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;/core/aggregation-pipeline-optimization
	&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Pipeline Optimization section: &lt;tt&gt;$skip + $limit&lt;/tt&gt; and &lt;tt&gt;$project + $skip + $limit&lt;/tt&gt; sequence optimization NO LONGER REORDER&lt;/li&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Pipeline Coalescence section: rewrite &lt;tt&gt;$sort + $limit coalescence&lt;/tt&gt;&lt;/li&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Update &lt;tt&gt;$sort + $skip + $limit&lt;/tt&gt; example to NO LONGER REORDER&lt;/li&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Update &lt;tt&gt;$limit + $skip + $limit + $skip&lt;/tt&gt; example  to NO LONGER REORDER&lt;/li&gt;
		&lt;li&gt;(meh) Add &lt;tt&gt;$sort + $unwind + $limit&lt;/tt&gt; example&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;/reference/operator/aggregation/sort
	&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Clarify behaviour change, specifically in the &lt;b&gt;$sort optimization + memory&lt;/b&gt; section&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;/reference/operator/aggregation/limit
	&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
		&lt;li&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Update note at the bottom of the page&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;


&lt;h2&gt;&lt;a name=&quot;Resources%28e.g.ScopeDocs%2CInvision%29&quot;&gt;&lt;/a&gt;Resources (e.g. Scope Docs, Invision)&lt;/h2&gt;

&lt;p&gt;My flowchart (attached)&lt;/p&gt;

&lt;p&gt; &lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;190287_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/190287/190287_IMG_0531.JPG&quot; title=&quot;IMG_0531.JPG&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;190287&quot; file-preview-title=&quot;IMG_0531.JPG&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/190287/_thumb_190287.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt; &lt;/p&gt;


&lt;h1&gt;&lt;a name=&quot;EngineeringTicketDescription%3A&quot;&gt;&lt;/a&gt;Engineering Ticket Description:&lt;/h1&gt;

&lt;p&gt;The new &lt;tt&gt;$project-$limit&lt;/tt&gt; optimization in 3.2 might make the pipeline to be split much earlier than before (because it will split the pipeline at the limit step).&lt;/p&gt;

&lt;p&gt;I&apos;m attaching two explain plan of queries, one which uses the optimization and one that doesn&apos;t because I added a &lt;tt&gt;$redact: $$KEEP&lt;/tt&gt; just before the &lt;tt&gt;$limit&lt;/tt&gt;.&lt;br/&gt;
In the case of this query much more fields are sent to the &lt;tt&gt;mergerPart&lt;/tt&gt; because of the splitting and is triggering a very bad behavior with second batches of aggregation queries which will be described in another ticket.&lt;/p&gt;

&lt;p&gt;I think it would be good to take into consideration pipeline splitting when doing those optimization (in addition there is no &lt;tt&gt;$sort&lt;/tt&gt; stage which would benefit from having the &lt;tt&gt;$limit&lt;/tt&gt; moved up)&lt;/p&gt;

&lt;p&gt;Cheers,&lt;br/&gt;
Antoine&lt;/p&gt;
</description>
                <environment></environment>
        <key id="469624">DOCS-11102</key>
            <summary>Docs for SERVER-24981: $project-$limit optimization has bad repercussion on pipeline splitting</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="allison.moore@mongodb.com">Allison Reinheimer Moore</assignee>
                                    <reporter username="kay.kim@mongodb.com">Kay Kim</reporter>
                        <labels>
                            <label>performance</label>
                    </labels>
                <created>Fri, 8 Dec 2017 00:39:41 +0000</created>
                <updated>Sun, 29 Oct 2023 13:58:41 +0000</updated>
                            <resolved>Tue, 24 Jul 2018 13:54:43 +0000</resolved>
                                                    <fixVersion>3.7.1</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="1955037" author="xgen-internal-githook" created="Tue, 24 Jul 2018 13:53:29 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;schmalliso&apos;, &apos;name&apos;: &apos;Allison Reinheimer Moore&apos;, &apos;email&apos;: &apos;allison.moore@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-11102&quot; title=&quot;Docs for SERVER-24981: $project-$limit optimization has bad repercussion on pipeline splitting&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-11102&quot;&gt;&lt;del&gt;DOCS-11102&lt;/del&gt;&lt;/a&gt;: clarify sort-limit coalescence behavior&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/48b2aa352ff4cf4af3bcbaafd24a491606fbe15b&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/48b2aa352ff4cf4af3bcbaafd24a491606fbe15b&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                            <outwardlinks description="documents">
                                        <issuelink>
            <issuekey id="299977">SERVER-24981</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="190287" name="IMG_0531.JPG" size="1775073" author="allison.moore@mongodb.com" created="Mon, 25 Jun 2018 20:42:15 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 26 Jun 2018 19:58:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 29 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>DOCS-11003</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 29 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>allison.moore@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kay.kim@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htltd3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htdfpb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1324">KANBAN BUCKET</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htlfh3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>