<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:31:31 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-75976] ShardingRecoveryService should provide blocking acquire API</title>
                <link>https://jira.mongodb.org/browse/SERVER-75976</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In the current ShardingRecoveryService API, &lt;em&gt;acquireRecoverableCriticalSectionBlockWrites&lt;/em&gt; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/sharding_recovery_service.cpp#L156&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;tasserts&lt;/a&gt; if the CS is held by some thread on the same namespace and expects the caller to check. This is problematic for two reasons:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;The caller only cares about acquiring CS and releasing it, it would need to know multiple additional APIs to check for others holding the CS.&lt;/li&gt;
	&lt;li&gt;The semantics of acquire/release in critical section APIs is usually block until acquire succeeds. Between checking for CS in IS mode and acquiring, there is a race and theoretically there is a starvation problem.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;ShardingRecoveryService can be improved to provide additional APIs to block until acquire and consolidate waiting/signals. This would also prevent incorrect usage bugs and simplify code. Although DDL operations are serialized at ShardingDDLCoordinator, there are cases like &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74880?focusedCommentId=5274234&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-5274234&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;these&lt;/a&gt; where acquiring CS can fail.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2311909">SERVER-75976</key>
            <summary>ShardingRecoveryService should provide blocking acquire API</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="sergi.mateo-bellido@mongodb.com">Sergi Mateo Bellido</assignee>
                                    <reporter username="abdul.qadeer@mongodb.com">Abdul Qadeer</reporter>
                        <labels>
                    </labels>
                <created>Tue, 11 Apr 2023 18:32:30 +0000</created>
                <updated>Fri, 12 May 2023 15:12:00 +0000</updated>
                            <resolved>Fri, 12 May 2023 15:12:00 +0000</resolved>
                                    <version>7.0.0-rc0</version>
                                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5420844" author="JIRAUSER1256927" created="Fri, 12 May 2023 15:12:00 +0000"  >&lt;p&gt;The proposed solution is to rely on another high level synchronization mechanism instead of using the Recoverable Critical Section. This is aligned with some discussions that we internally had at Sharding EMEA.&lt;/p&gt;</comment>
                            <comment id="5420782" author="JIRAUSER1269163" created="Fri, 12 May 2023 14:53:55 +0000"  >&lt;p&gt;That makes sense, thanks for the explanation. I am fine with the solution you proposed.&lt;/p&gt;</comment>
                            <comment id="5419850" author="JIRAUSER1256927" created="Fri, 12 May 2023 07:15:42 +0000"  >&lt;p&gt;The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/75ef59b570f937c0a14944187d43bf0707543980/src/mongo/db/s/shard_server_op_observer.cpp#L208-L239&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;in-memory CS section is acquired as part of the commit of the WUOW associated with the write of the recoverable CS&lt;/a&gt;.&#160;&lt;/p&gt;

&lt;p&gt;About resharding, if it didn&apos;t have any top level synchronization it would be hitting the same kind of error. More specifically, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/75ef59b570f937c0a14944187d43bf0707543980/src/mongo/db/s/reshard_collection_coordinator.h#L37-L39&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;resharding spawns a DDL Coordinator on the primary shard of the &lt;tt&gt;dbName&lt;/tt&gt; before reaching the CSRS&lt;/a&gt;. This prevents the execution of any other DDL that could potentially acquire the CS, except for migrations. Then, for migrations, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/75ef59b570f937c0a14944187d43bf0707543980/src/mongo/db/s/resharding/resharding_coordinator_service.cpp#L345&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the resharding coordinator service on the CSRS disallows them during the reshard op&lt;/a&gt;. Thus, a this point we are sure that no one else can use the CS.&lt;/p&gt;

&lt;p&gt;We did a hacky temporary solution for the &lt;tt&gt;movePrimary&lt;/tt&gt;, but we would like to avoid exposing more synchronization levels unless it is unavoidable.&#160; Happy to jump in a zoom if you wanna discuss further &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/biggrin.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="5419093" author="JIRAUSER1269163" created="Thu, 11 May 2023 20:42:30 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sergi.mateo-bellido%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;sergi.mateo-bellido@mongodb.com&quot;&gt;sergi.mateo-bellido@mongodb.com&lt;/a&gt; I am not adding any in-memory CS with CollectionShardingRuntime before interacting with ShardingRecoveryService explicitly - not sure if you mean something implicit happening in the bg. Per &lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/s/sharding_recovery_service.cpp#L216[]&quot; title=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/s/sharding_recovery_service.cpp#L216&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this&lt;/a&gt; comment in the API, I was under the impression that an in-memory CS is acquired after persisting (which makes sense to persist before updating in-memory). The critical section is also taken currently by the resharding &lt;a href=&quot;https://github.com/10gen/mongo/blob/a732d6f8067c089e27a293a01e47e1526709626d/src/mongo/db/s/resharding/resharding_recipient_service.cpp#L776&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;recipient&lt;/a&gt; and &lt;a href=&quot;https://github.com/10gen/mongo/blob/a732d6f8067c089e27a293a01e47e1526709626d/src/mongo/db/s/resharding/resharding_donor_service.cpp#L705&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;donor&lt;/a&gt; both, and both don&apos;t spawn a DDL coordinator. I am not sure if spawning a coordinator in all these places is feasible.&lt;/p&gt;</comment>
                            <comment id="5408302" author="JIRAUSER1256927" created="Mon, 8 May 2023 14:33:28 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=abdul.qadeer%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;abdul.qadeer@mongodb.com&quot;&gt;abdul.qadeer@mongodb.com&lt;/a&gt; what do you think? We discussed some time ago about this issue, I guess that at some point we will tackle but rather than fixing it at the level of the critical section we will do it trying to spawn a coordinator also on the recipient.&lt;/p&gt;</comment>
                            <comment id="5408296" author="JIRAUSER1256927" created="Mon, 8 May 2023 14:31:31 +0000"  >&lt;p&gt;The &lt;tt&gt;Recoverable Critical Section&lt;/tt&gt; is just an utility that added persistence to the already existing &lt;tt&gt;in-memory critical section&lt;/tt&gt; (i.e. the one you obtain through the &lt;tt&gt;CollectionShardingRuntime&lt;/tt&gt;). Note that the &lt;tt&gt;recoverable critical section&lt;/tt&gt; keeps synchronized the persisted representation of the critical section with the in-memory one, so internally all &lt;tt&gt;recoverable critical sections&lt;/tt&gt; are also &lt;tt&gt;in-memory critical sections&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;In terms of synchronization, the &lt;tt&gt;Recoverable Critical Section&lt;/tt&gt; is also similar to the in-memory one: their acquisition happen in a context in which it should be impossible that the critical section was already acquired by someone else. This is usually guaranteed through the usage of a higher level synchronization mechanism (e.g. DDL Coordinators that stop migrations). Note that in general we don&apos;t want to wait until the CS is released to do something: in most of our cases what we want is to throw an exception if the critical section is held! You have to imagine the critical section as the last resource you need to acquire to modify the shard authoritative metadata. If someone else is already modifying that metadata, you don&apos;t really know what&apos;s going to happen to the collection: perhaps the shard will get a new chunk, perhaps the collection is renamed or, even worse, perhaps it is dropped.&lt;/p&gt;

&lt;p&gt;For your specific case, I think that the real issue is that the recipient shard of a &lt;tt&gt;movePrimary&lt;/tt&gt; doesn&apos;t spawn a DDL Coordinator. Thus, the higher level synchronization mechanism I mentioned above is not present.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="2311920">SERVER-75977</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25133"><![CDATA[Sharding EMEA]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Apr 2023 09:54:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        38 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>sergi.mateo-bellido@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            38 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>abdul.qadeer@mongodb.com</customfieldvalue>
            <customfieldvalue>sergi.mateo-bellido@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i248fz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1mk80:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7186">Sharding EMEA 2023-05-15</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i23ulb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>