<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:49:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-60420] The slow &apos;shardCollection&apos; path performs heavy work under the critical section</title>
                <link>https://jira.mongodb.org/browse/SERVER-60420</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;As part of the &quot;Robust DDL&quot; changes (PM-1965), the amount of work which &lt;tt&gt;shardCollection&lt;/tt&gt; does under the critical section has increased from 4.4 to 5.0 and this leads to writes being blocked for longer time than necessary:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;b&gt;In 4.4:&lt;/b&gt; The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f8f7ff1dc6d21c304d1c0b095bde7ac40c018904/src/mongo/db/s/shardsvr_shard_collection.cpp#L612&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;indexes and chunks&lt;/a&gt; are written outside of the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f8f7ff1dc6d21c304d1c0b095bde7ac40c018904/src/mongo/db/s/shardsvr_shard_collection.cpp#L565&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;write part&lt;/a&gt; of the critical section&lt;/li&gt;
	&lt;li&gt;&lt;b&gt;In 5.0:&lt;/b&gt; We do not free the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2d64fa08148b302eae3565123a1752bb895bfbaf/src/mongo/db/s/create_collection_coordinator.cpp#L451&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;critical section&lt;/a&gt; while &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2d64fa08148b302eae3565123a1752bb895bfbaf/src/mongo/db/s/create_collection_coordinator.cpp#L475&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;creating the indexes&lt;/a&gt; and that&apos;s what is causing &lt;tt&gt;shardCollection&lt;/tt&gt; to take blocking time proportional to the size of the collection.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;For more context, if a collection is being sharded and we discover that that collection is empty (has zero documents), we hold the critical section for the entire duration (index creation, chunks creation, etc.) in order to ensure that no writes come in case we are creating chunks on shards other than the primary.&lt;/p&gt;

&lt;p&gt;However, if we discover that the collection has documents, we can only create chunks locally and therefore we can allow writes to proceed because they will always be searching on the primary shard.&lt;/p&gt;

&lt;p&gt;As part of this ticket we should do the absolutely minimum amount of work under the critical section, as part of the optimised path.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1888937">SERVER-60420</key>
            <summary>The slow &apos;shardCollection&apos; path performs heavy work under the critical section</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="marcos.grillo@mongodb.com">Marcos Jos&#233; Grillo Ramirez</assignee>
                                    <reporter username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</reporter>
                        <labels>
                    </labels>
                <created>Mon, 4 Oct 2021 13:21:15 +0000</created>
                <updated>Sun, 29 Oct 2023 21:47:52 +0000</updated>
                            <resolved>Fri, 15 Oct 2021 11:47:01 +0000</resolved>
                                    <version>5.0.3</version>
                    <version>5.1.0-rc0</version>
                                    <fixVersion>5.2.0</fixVersion>
                    <fixVersion>5.0.4</fixVersion>
                    <fixVersion>5.1.0-rc1</fixVersion>
                                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="4129233" author="xgen-internal-githook" created="Mon, 18 Oct 2021 13:42:12 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Marcos Jos&#233; Grillo Ramirez&apos;, &apos;email&apos;: &apos;marcos.grillo@mongodb.com&apos;, &apos;username&apos;: &apos;m4nti5&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-60420&quot; title=&quot;The slow &amp;#39;shardCollection&amp;#39; path performs heavy work under the critical section&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-60420&quot;&gt;&lt;del&gt;SERVER-60420&lt;/del&gt;&lt;/a&gt; Release the critical section early in the shard collection slow path&lt;/p&gt;

&lt;p&gt;(cherry picked from commit e7c0fc2c3eac5d5fd35b29fcc7d1d5d52946c293)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/827838d784cf439d84cdb3b467fe812be9cf034c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/827838d784cf439d84cdb3b467fe812be9cf034c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4129169" author="xgen-internal-githook" created="Mon, 18 Oct 2021 13:22:47 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Marcos Jos&#233; Grillo Ramirez&apos;, &apos;email&apos;: &apos;marcos.grillo@mongodb.com&apos;, &apos;username&apos;: &apos;m4nti5&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-60420&quot; title=&quot;The slow &amp;#39;shardCollection&amp;#39; path performs heavy work under the critical section&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-60420&quot;&gt;&lt;del&gt;SERVER-60420&lt;/del&gt;&lt;/a&gt; Release the critical section early in the shard collection slow path&lt;/p&gt;

&lt;p&gt;(cherry picked from commit e7c0fc2c3eac5d5fd35b29fcc7d1d5d52946c293)&lt;br/&gt;
Branch: v5.1&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/0c95864a269d7a66223eafe07906f8c87dacc1b7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/0c95864a269d7a66223eafe07906f8c87dacc1b7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4126113" author="xgen-internal-githook" created="Fri, 15 Oct 2021 11:44:04 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Marcos Jos&#233; Grillo Ramirez&apos;, &apos;email&apos;: &apos;marcos.grillo@mongodb.com&apos;, &apos;username&apos;: &apos;m4nti5&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-60420&quot; title=&quot;The slow &amp;#39;shardCollection&amp;#39; path performs heavy work under the critical section&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-60420&quot;&gt;&lt;del&gt;SERVER-60420&lt;/del&gt;&lt;/a&gt; Release the critical section early in the shard collection slow path&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e7c0fc2c3eac5d5fd35b29fcc7d1d5d52946c293&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e7c0fc2c3eac5d5fd35b29fcc7d1d5d52946c293&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4121569" author="marcos.grillo" created="Wed, 13 Oct 2021 14:43:18 +0000"  >&lt;p&gt;The sharded index creation library verifies &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d9d70aaf89337e3e6a263abcc271d159fe857985/src/mongo/db/s/shard_key_util.cpp#L227&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;if the collection is empty&lt;/a&gt; before &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d9d70aaf89337e3e6a263abcc271d159fe857985/src/mongo/db/s/shard_key_util.cpp#L195&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;starting the creation&lt;/a&gt;, and this verification did not change in 5.0, so, the index creation cannot happen inside of the critical section in the slow path, however, we do create chunks now under the critical section, as reported in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-60609&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;SERVER-60609&lt;/a&gt;, I&apos;ll close &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-60609&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;SERVER-60609&lt;/a&gt; as a duplicate of this ticket, because in the description of this ticket we already specify that we&apos;ll minimize the amount of work done in the critical section.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="1895474">SERVER-60609</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                                                <inwardlinks description="is caused by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="22495"><![CDATA[v5.1]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 11 Oct 2021 22:39:15 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 16 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 16 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>marcos.grillo@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i04407:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyqncn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5425">Sharding EMEA 2021-10-18</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i03q5j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>