<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:28:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-31902] Reduce contention on ShardingState::_mutex for insert workloads</title>
                <link>https://jira.mongodb.org/browse/SERVER-31902</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently, each write that&apos;s part of a batch performs two separate checks for shard version. Once at the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/271879b7a67c9d9b36778692f2a77e04c6403a1f/src/mongo/db/ops/write_ops_exec.cpp#L177&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;beginning&lt;/a&gt; of the batch and a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/271879b7a67c9d9b36778692f2a77e04c6403a1f/src/mongo/db/s/collection_sharding_state.cpp#L314&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;second time&lt;/a&gt; when the write has been written to the oplog.&lt;/p&gt;

&lt;p&gt;Performing these checks causes a hot mutex to be acquired in order to retrieve the sharding state for the collection, so the first one should ideally be removed.&lt;/p&gt;</description>
                <environment></environment>
        <key id="457004">SERVER-31902</key>
            <summary>Reduce contention on ShardingState::_mutex for insert workloads</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</assignee>
                                    <reporter username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</reporter>
                        <labels>
                    </labels>
                <created>Fri, 10 Nov 2017 09:04:24 +0000</created>
                <updated>Mon, 11 Dec 2017 22:14:18 +0000</updated>
                            <resolved>Fri, 17 Nov 2017 15:32:47 +0000</resolved>
                                    <version>3.6.0-rc3</version>
                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1728862" author="kaloian.manassiev" created="Fri, 17 Nov 2017 15:32:47 +0000"  >&lt;p&gt;Unfortunately, given the current state of the code it is really difficult to reduce the number of retrievals of the collection metadata on the write path to less than 2. The more proper solution would be to make &lt;tt&gt;CollectionShardingState&lt;/tt&gt; be a decoration on the &lt;tt&gt;Collection&lt;/tt&gt; object, which has already been retrieved under an IX lock. This would eliminate the mutex acquisition altogether.&lt;/p&gt;

&lt;p&gt;Closing as duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-25061&quot; title=&quot;Make CollectionShardingState a decoration on Collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-25061&quot;&gt;&lt;del&gt;SERVER-25061&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1724084" author="milkie" created="Mon, 13 Nov 2017 16:26:18 +0000"  >&lt;p&gt;I changed the title to reflect the problem to solve rather than suggesting a solution.&lt;/p&gt;</comment>
                            <comment id="1723984" author="milkie" created="Mon, 13 Nov 2017 15:02:39 +0000"  >&lt;p&gt;I am adding the dependency back in because regaining the performance depends on the work here, not because this ticket identifies a cause of the regression from a previous version of MongoDB.&lt;/p&gt;</comment>
                            <comment id="1723980" author="kaloian.manassiev" created="Mon, 13 Nov 2017 14:58:32 +0000"  >&lt;blockquote&gt;&lt;p&gt;It sounds like we can solve this by removing the shard version check in CollectionShardingState::onInsertOp(), since it should already have been checked when the db lock was acquired.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;This will not work exactly because of the yielding problem. If the query part of an update yields and the shard version changes, if we don&apos;t check inside of &lt;tt&gt;onInsertOp&lt;/tt&gt; we will lose that write. In either case, these two shard version checks have always been around so they should not have contributed to the performance regression in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31694&quot; title=&quot;17% throughput regression in insert workload&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31694&quot;&gt;&lt;del&gt;SERVER-31694&lt;/del&gt;&lt;/a&gt;, so I am removing the &quot;dependency&quot;.&lt;/p&gt;

&lt;p&gt;I am pretty sure the query system doesn&apos;t take locks itself (outside of re-acquisition after yield). It relies on being called locked. Also &lt;tt&gt;AutoGetCollectionForReadCommand&lt;/tt&gt; is only intended to be used for read commands.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;If we are not checking shard version after recovering from yield, that would be a bug, but I feel like we did indeed do this at one point. Did the logic get removed?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I think we did code inspection at some point and deemed that the check in the OpObserver is sufficient, but on second review it is not because of the scenario I described.&lt;/p&gt;</comment>
                            <comment id="1723920" author="milkie" created="Mon, 13 Nov 2017 13:41:44 +0000"  >&lt;p&gt;AutoGetCollectionForReadCommand checks for shard version, but I&apos;m not sure if that&apos;s what the query system uses for the query phase of update.&lt;/p&gt;</comment>
                            <comment id="1723919" author="milkie" created="Mon, 13 Nov 2017 13:39:02 +0000"  >&lt;p&gt;It sounds like we can solve this by removing the shard version check in CollectionShardingState::onInsertOp(), since it should already have been checked when the db lock was acquired.&lt;br/&gt;
If we are not checking shard version after recovering from yield, that would be a bug, but I feel like we did indeed do this at one point. Did the logic get removed?&lt;/p&gt;</comment>
                            <comment id="1723902" author="kaloian.manassiev" created="Mon, 13 Nov 2017 12:54:48 +0000"  >&lt;p&gt;Unfortunately, in order to cover the case where a write operation does not result in an entry being written to the oplog, it is not possible to remove the check performed at the beginning of the write batch. Otherwise, writes which result in a no-op will never discover that the shard version has changed.&lt;/p&gt;

&lt;p&gt;In the same train of thought, I noticed that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c298f081852f1fc1ae4a51ed8997bd0fd15bb5aa/src/mongo/db/query/query_yield.cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;returning from yield&lt;/a&gt; doesn&apos;t check that the shard version has been preserved. In this case, it is possible that if a chunk moves out of a shard while the query for a write operation has yielded, then updates will potentially not match their predicate, result in a no-op and report success, while in fact the document still exists.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt;: Do we need to add a check for shard version after returning from yielding as well?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="449967">SERVER-31694</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="301086">SERVER-25061</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 13 Nov 2017 13:39:02 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 12 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_14262" key="com.atlassian.jira.plugin.system.customfieldtypes:datepicker">
                        <customfieldname>End date</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 10 Nov 2017 23:59:59 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 12 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htjpdz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htbfzr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_14261" key="com.atlassian.jira.plugin.system.customfieldtypes:datepicker">
                        <customfieldname>Start date</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 10 Nov 2017 00:00:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htjbhr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>