<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:00:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-42431] New config server primary unlocks all distlocks held by previous config server on stepup</title>
                <link>https://jira.mongodb.org/browse/SERVER-42431</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Config servers all &lt;a href=&quot;https://github.com/mongodb/mongo/blob/917d338c4bf52dc8dce2c0e585a676385e81ed1c/src/mongo/s/sharding_initialization.h#L61&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;have the same process id of &quot;ConfigServer.&quot;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;On transition to primary, a config node &lt;a href=&quot;https://github.com/mongodb/mongo/blob/917d338c4bf52dc8dce2c0e585a676385e81ed1c/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L764&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;unlocks all existing distlocks with the process id &quot;ConfigServer.&quot;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This means that DDL operations which serialize on the config server via a distlock but whose business logic is executed by a shard (moveChunk, movePrimary, and shardCollection) are suspect, because the shard can keep executing the business logic outside the distlock. For example, you could drop a database concurrently with sharding a collection and end up with a config.collections entry without a corresponding config.databases entry.&lt;/p&gt;

&lt;p&gt;Note that the track unsharded project will add two more DDL operations with the shardCollection pattern (renameCollection and convertToCapped).&lt;/p&gt;</description>
                <environment></environment>
        <key id="875956">SERVER-42431</key>
            <summary>New config server primary unlocks all distlocks held by previous config server on stepup</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="backlog-server-sharding-emea">[DO NOT USE] Backlog - Sharding EMEA</assignee>
                                    <reporter username="esha.maharishi@mongodb.com">Esha Maharishi</reporter>
                        <labels>
                            <label>sharding-DDL-bugs</label>
                    </labels>
                <created>Thu, 25 Jul 2019 20:12:12 +0000</created>
                <updated>Fri, 27 Oct 2023 20:42:45 +0000</updated>
                            <resolved>Thu, 10 Feb 2022 09:26:41 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="4347202" author="kaloian.manassiev" created="Thu, 10 Feb 2022 09:26:41 +0000"  >&lt;p&gt;This is now Gone Away after 5.0, because under the DDL project we have local synchronisation between DDL and moveChunk, which was the main reason for having the config server dist lock.&lt;/p&gt;</comment>
                            <comment id="2348713" author="esha.maharishi@10gen.com" created="Mon, 29 Jul 2019 19:31:00 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt;, hmm, I think the sharding catalog can only get corrupted by metadata commands for which the &lt;em&gt;shard&lt;/em&gt; directly writes to the sharding catalog. So, I think it is only a problem for shardCollection, movePrimary, and moveChunk. This is because &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.2.0-rc4/src/mongo/db/db.cpp#L556&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;config servers use ConnectionString::forLocal&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.2.0-rc4/src/mongo/db/s/sharding_initialization_mongod.cpp#L396&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;for ConnectionString::LOCAL, the ShardRegistry returns ShardLocal instances&lt;/a&gt;. So, if the config server is the one that writes to the sharding catalog, it will write on the same branch of history as the distlock was on.&lt;/p&gt;

&lt;p&gt;The problem is if a &lt;em&gt;shard&lt;/em&gt; targets the new config primary, and therefore updates the sharding catalog on the new branch of history that has released the distlock.&lt;/p&gt;

&lt;p&gt;Note that this second issue:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Another way this can manifest that would result in actual user data loss is if an old config primary executed dropCollection after the collection had been recreated on the new config primary.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;would not be solved even if the new config primary reacquired persisted locks.&lt;/p&gt;</comment>
                            <comment id="2348616" author="kaloian.manassiev" created="Mon, 29 Jul 2019 18:54:15 +0000"  >&lt;p&gt;When we moved the balancer to the config server in 3.4, the collection lock acquisition due to moveChunk was the only operation, which was taking distributed locks on the config server. At the time, we must have (knowingly or not) made the decision that it would be cleaner to have step-up clean-up these locks so that (1) the migration manager recovery doesn&apos;t get stuck and (2) because the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69c124b4065232c8a97af3104f40d0b4806415b/src/mongo/db/s/balancer/migration_manager.cpp#L211&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;migration manager recovery&lt;/a&gt; will re-acquire them.&lt;/p&gt;

&lt;p&gt;I guess in 3.6 we moved more operations to the config server, which didn&apos;t have the recovery process that moveChunk goes through and because of this we introduced this bug.&lt;/p&gt;

&lt;p&gt;Given that the lock manager will re-acquire locks with the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69c124b4065232c8a97af3104f40d0b4806415b/src/mongo/db/s/balancer/migration_manager.h#L256&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;same session id&lt;/a&gt;, we could remove this behaviour where the lock manager removes all locks by making the &lt;tt&gt;MigrationManager&lt;/tt&gt; have a fixed session id and on step-up only unlocking locks acquired by that session id. This will preserve the intra-node synchronization behaviour we rely on through the dist locks, without requiring us to build a more sophisticated dist lock manager.&lt;/p&gt;</comment>
                            <comment id="2344725" author="jason.zhang" created="Thu, 25 Jul 2019 22:29:26 +0000"  >&lt;p&gt;Attached is a repro. For consistent repro you need to change &lt;a href=&quot;https://github.com/mongodb/mongo/blob/284630e3704be4be091154c5b7fe63c335e9d70c/src/mongo/s/client/shard.cpp#L46&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongos&apos;s internal retries&lt;/a&gt;&#160;to 1. This is because after stepping down the config server, mongos will retry &apos;shardCollection&apos; on the new config server, which will take distlocks that will block dropCollection from taking distlocks on the target database.&lt;/p&gt;</comment>
                            <comment id="2344561" author="esha.maharishi@10gen.com" created="Thu, 25 Jul 2019 20:13:08 +0000"  >&lt;p&gt;Assigned to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jason.zhang&quot; class=&quot;user-hover&quot; rel=&quot;jason.zhang&quot;&gt;jason.zhang&lt;/a&gt; to write a repro that demonstrates the sharding catalog becoming inconsistent if the config server fails over during shardCollection.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="225240" name="configsvrStepdownRaceRepro.txt" size="589176" author="jason.zhang@mongodb.com" created="Thu, 25 Jul 2019 22:26:16 +0000"/>
                            <attachment id="225239" name="configsvr_failover_repro.js" size="1901" author="jason.zhang@mongodb.com" created="Thu, 25 Jul 2019 22:23:06 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25133"><![CDATA[Sharding EMEA]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 25 Jul 2019 22:29:26 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 51 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 51 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-sharding-emea</customfieldvalue>
            <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>jason.zhang@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvhkj3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr6qyf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3062">Sharding 2019-07-29</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvh6sf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>