<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:18:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-71328] Ensure correct filtering metadata on donor shard after multiple failures </title>
                <link>https://jira.mongodb.org/browse/SERVER-71328</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The donor of a chunk migration calls &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L732&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;ShardingStateRecovery::endMetadataOp()&lt;/tt&gt;&lt;/a&gt; that is persisting the configOpTime inclusive of the migration commit, this is to ensure that in case of stepdown when the next primary node will read from the config server it will see the effect of the commit performed by the previous primary.&lt;/p&gt;

&lt;p&gt;The problem is that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L732&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;endMetadataOp()&lt;/tt&gt;&lt;/a&gt; is not called after recovering a failed migration, so in case the donor experiences an error during the commit (network error) and a subsequent stepdown, there is no guarantee that the next primary node will install the correct filtering metadata inclusive of the last migration.&lt;/p&gt;

&lt;p&gt;The proposed solution is to add a VectorClock::waitForDurableConfigTime() &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d4cd973ae39fe2625a84f2f7b711f9c7ebe885e9/src/mongo/db/s/migration_coordinator.cpp#L174-L183&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;just before writing down the commit decision in the migration coordinator document.&lt;/a&gt;&lt;br/&gt;
This will be execute both if no error occur during the commit as well as during migration recovery.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2184611">SERVER-71328</key>
            <summary>Ensure correct filtering metadata on donor shard after multiple failures </summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="tommaso.tocci@mongodb.com">Tommaso Tocci</assignee>
                                    <reporter username="tommaso.tocci@mongodb.com">Tommaso Tocci</reporter>
                        <labels>
                            <label>sharding-wfbf-day</label>
                    </labels>
                <created>Mon, 14 Nov 2022 16:59:03 +0000</created>
                <updated>Sun, 29 Oct 2023 21:30:37 +0000</updated>
                            <resolved>Fri, 6 Jan 2023 01:04:29 +0000</resolved>
                                    <version>5.0.13</version>
                    <version>6.0.2</version>
                    <version>6.1.0-rc4</version>
                    <version>6.2.0-rc0</version>
                                    <fixVersion>6.3.0-rc0</fixVersion>
                    <fixVersion>6.0.5</fixVersion>
                    <fixVersion>5.0.16</fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5214885" author="xgen-internal-githook" created="Tue, 21 Feb 2023 00:05:42 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Tommaso Tocci&apos;, &apos;email&apos;: &apos;tommaso.tocci@mongodb.com&apos;, &apos;username&apos;: &apos;toto-dev&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-71328&quot; title=&quot;Ensure correct filtering metadata on donor shard after multiple failures &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-71328&quot;&gt;&lt;del&gt;SERVER-71328&lt;/del&gt;&lt;/a&gt; Ensure correct filtering metadata on donor shard after multiple failures&lt;/p&gt;

&lt;p&gt;(cherry picked from commit f782ca31ce08c7b0ec04a29eac3338884338a337)&lt;br/&gt;
(cherry picked from commit fd8260fc72cbe43f4addeab7bada2a87b05fb28c)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/ef86c9f8322eaef228c36cebc9a990686dff3a40&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/ef86c9f8322eaef228c36cebc9a990686dff3a40&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5108715" author="xgen-internal-githook" created="Thu, 12 Jan 2023 12:43:33 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;toto&apos;, &apos;email&apos;: &apos;58224719+toto-dev@users.noreply.github.com&apos;, &apos;username&apos;: &apos;toto-dev&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-71328&quot; title=&quot;Ensure correct filtering metadata on donor shard after multiple failures &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-71328&quot;&gt;&lt;del&gt;SERVER-71328&lt;/del&gt;&lt;/a&gt; Ensure correct filtering metadata on donor shard after multiple failures&lt;/p&gt;

&lt;p&gt;(cherry picked from commit f782ca31ce08c7b0ec04a29eac3338884338a337)&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/fd8260fc72cbe43f4addeab7bada2a87b05fb28c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/fd8260fc72cbe43f4addeab7bada2a87b05fb28c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5093773" author="xgen-internal-githook" created="Thu, 5 Jan 2023 18:46:12 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;toto&apos;, &apos;email&apos;: &apos;58224719+toto-dev@users.noreply.github.com&apos;, &apos;username&apos;: &apos;toto-dev&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-71328&quot; title=&quot;Ensure correct filtering metadata on donor shard after multiple failures &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-71328&quot;&gt;&lt;del&gt;SERVER-71328&lt;/del&gt;&lt;/a&gt; Ensure correct filtering metadata on donor shard after multiple failures&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f782ca31ce08c7b0ec04a29eac3338884338a337&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f782ca31ce08c7b0ec04a29eac3338884338a337&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25041"><![CDATA[v6.2]]></customfieldvalue>
    <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 7 Dec 2022 20:05:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        50 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            50 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>tommaso.tocci@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1ifen:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i13dro:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6663">Sharding EMEA 2022-12-12</customfieldvalue>
    <customfieldvalue id="6664">Sharding EMEA 2022-12-26</customfieldvalue>
    <customfieldvalue id="6810">Sharding EMEA 2023-01-09</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;ul&gt;
	&lt;li&gt;Migration starts on current primary node P0 of the donor shard.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L476&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Commit of the migration fail due to network error&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L485&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_cleanup()&lt;/a&gt; is executed and&#160; as part of it &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L732&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;endMetadataOp()&lt;/tt&gt;&lt;/a&gt; will persist the latest config time. This is useless since we still don&apos;t have knowledge of the config time that is inclusive of the migration commit.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L486&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Asynchronous recovery of the migration is spawned&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;During recovery we read again from the config server and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_util.cpp#L1081&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;we realized that actually the commit succeed &lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;We will call &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_util.cpp#L1088&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;completeMigration()&lt;/a&gt; and we will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d4cd973ae39fe2625a84f2f7b711f9c7ebe885e9/src/mongo/db/s/migration_coordinator.cpp#L180&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;persist the migration decision in the coordinator document&lt;/a&gt; &lt;b&gt;without calling &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L732&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;endMetadataOp()&lt;/tt&gt;&lt;/a&gt;&lt;/b&gt;.&lt;/li&gt;
	&lt;li&gt;Stepdown will happen before removing the coordinator document.&lt;/li&gt;
	&lt;li&gt;A new primary node P1 of the donor shard will be elected and it will try to recover the migration again since the coordinator document is still present.&lt;/li&gt;
	&lt;li&gt;This time it will find that the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_util.cpp#L1000-L1002&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;migration decision have been already set as kCommitted&lt;/a&gt; and it will call again &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_util.cpp#L1088&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;completeMigration()&lt;/a&gt; &lt;b&gt;without calling &lt;a href=&quot;https://github.com/mongodb/mongo/blob/93612bf95f8003a1e1d6af27fb1b67c57c6f3e39/src/mongo/db/s/migration_source_manager.cpp#L732&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;endMetadataOp()&lt;/tt&gt;&lt;/a&gt;&lt;/b&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Since the configOpTime known to P1 is not inclusive of the latest committed migration, there is no guarantee that any subsequent refresh of the filtering metadata would include the committed migration.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1i1jz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>