<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:14:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-47563] Add concurrency control to the CollectionShardingState class such that it no longer depends upon collection locks for concurrency control</title>
                <link>https://jira.mongodb.org/browse/SERVER-47563</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Execution&apos;s Lock Free Reads project will allow reads to run without collection and database intent (MODE_IS) locks. The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a94df7c8c6f36e67086406b5778368f97dc1bd7f/src/mongo/db/s/collection_sharding_state.h#L55-L57&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CollectionShardingState class currently relies on collection locks for concurrency control&lt;/a&gt;. The implementations of CollectionShardingState must create their own read concurrency control mechanisms that do not rely on reads taking collection IS locks.&lt;/p&gt;

&lt;p&gt;The CollectionShardingRuntime (an implementation of CSS) appears to already have a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a94df7c8c6f36e67086406b5778368f97dc1bd7f/src/mongo/db/s/collection_sharding_runtime.h#L60&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CSRLock&lt;/a&gt; that can be taken in shared and exclusive mode: perhaps its use simply needs to be expanded.&lt;/p&gt;

&lt;p&gt;The two areas of the CollectionShardingRuntime concurrency that appear to be of particular interest are:&lt;/p&gt;

&lt;p&gt;1. Installing new and fetching the current routing table information (CollectionMetadata) for callers.&lt;/p&gt;

&lt;p&gt;2. The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a94df7c8c6f36e67086406b5778368f97dc1bd7f/src/mongo/db/s/sharding_migration_critical_section.cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ShardingMigrationCriticalSection for each collection has no concurrency control itself&lt;/a&gt;. It depends upon the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a94df7c8c6f36e67086406b5778368f97dc1bd7f/src/mongo/db/s/collection_sharding_runtime.cpp#L181-L197&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CollectionShardingRuntime&lt;/a&gt; to provide concurrency control. The two ShardingMigrationCriticalSection::entering* functions expect a collection X lock, the ShardingMigrationCriticalSection::exit* function expects a IX lock, and ShardingMigrationCriticalSection::get* presumably expects callers to hold a collection IS lock.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1316155">SERVER-47563</key>
            <summary>Add concurrency control to the CollectionShardingState class such that it no longer depends upon collection locks for concurrency control</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="backlog-server-sharding">[DO NOT USE] Backlog - Sharding Team</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                    </labels>
                <created>Wed, 15 Apr 2020 17:02:15 +0000</created>
                <updated>Fri, 27 Oct 2023 20:42:15 +0000</updated>
                            <resolved>Mon, 20 Jul 2020 15:14:32 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3292458" author="dianna.hohensee" created="Mon, 20 Jul 2020 15:14:32 +0000"  >&lt;p&gt;With the resolution of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47566&quot; title=&quot;CollectionShardingRuntime::getCriticalSectionSignal accesses _critSec without locks&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47566&quot;&gt;&lt;del&gt;SERVER-47566&lt;/del&gt;&lt;/a&gt;, I believe the sharding state has the concurrency control it needs in order to work properly without user reads holding MODE_IS coll/db locks.&lt;/p&gt;</comment>
                            <comment id="3064302" author="dianna.hohensee" created="Fri, 1 May 2020 13:19:36 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt;, I believe we determined that sharding has no more work (after &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47564&quot; title=&quot;Add concurrency control to setting and unsetting the ShardingMigrationCriticalSection flag that does not rely upon collection locks&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47564&quot;&gt;&lt;del&gt;SERVER-47564&lt;/del&gt;&lt;/a&gt;) to do here. Is it alright to close this?&lt;/p&gt;

&lt;p&gt;Kal and I discussed the CollectionShardingState lifetime compared to Collection and determined that it is safe for now and known planned changes, so we are not going to make CSS a decoration of Collection.&lt;/p&gt;</comment>
                            <comment id="3042488" author="dianna.hohensee" created="Thu, 16 Apr 2020 13:49:52 +0000"  >&lt;p&gt;I think I agree that the CollectionShardingState is concurrency safe due to mutexes, from what I can figure out. It looks like the _metadataManagerLock takes care of concurrency for fetching and setting the routing information, and the CSRLock protects the critical section reads/writes. &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47566&quot; title=&quot;CollectionShardingRuntime::getCriticalSectionSignal accesses _critSec without locks&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47566&quot;&gt;&lt;del&gt;SERVER-47566&lt;/del&gt;&lt;/a&gt; still needs to fix a concurrency bug protecting the critical section, though.&lt;/p&gt;

&lt;p&gt;I&apos;ll look into the decoration idea. I think I need to more details on what happens between fetching the CollectionShardingState and the Collection instances for an operation, to see what could go wrong without the atomicity of a MODE_X collection lock.&lt;/p&gt;</comment>
                            <comment id="3041910" author="kaloian.manassiev" created="Thu, 16 Apr 2020 05:41:54 +0000"  >&lt;p&gt;I don&apos;t think this is necessary - the CSRLock already serves this purpose, so we should be good there, but I think the returned &lt;tt&gt;CollectionShardingState&lt;/tt&gt; is a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0793797e87ea2c9c6e0a92f5fd704da15ffc0e51/src/mongo/db/s/collection_sharding_state.cpp#L122&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;direct pointer&lt;/a&gt; into a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0793797e87ea2c9c6e0a92f5fd704da15ffc0e51/src/mongo/db/s/collection_sharding_state.cpp#L107&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;map&lt;/a&gt;, which intends to parallel the &lt;tt&gt;CollectionCatalog&lt;/tt&gt;. Currently, we never delete from this map and rely that around drop and create, the collection X lock will ensure a barrier between the transitions of the objects there. However, with lock-free reads, this may be a bit hard to reason.&lt;/p&gt;

&lt;p&gt;Since I see that the &lt;tt&gt;Collection&lt;/tt&gt; class is already decorable, can I propose that we get rid of that map and put the &lt;tt&gt;CollectionShardingState&lt;/tt&gt; as a decoration on &lt;tt&gt;Collection&lt;/tt&gt;? That way we don&apos;t need to worry about the lifetime of the &lt;tt&gt;CSS&lt;/tt&gt; and the CSRLock is only for the state of the CSS.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="1316215">SERVER-47566</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="1316170">SERVER-47564</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25141"><![CDATA[Sharding]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 16 Apr 2020 05:41:54 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 29 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-47566'>SERVER-47566</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1527</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 29 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-sharding</customfieldvalue>
            <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxfhef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hx2z1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxf3nr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>