<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:10:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-45916] On primary, 2-phase index build cleanup writes an abortIndexBuild oplog entry under a stronger mode user collection lock X which can lead to 3 way deadlock with prepared transactions, step down and index build</title>
                <link>https://jira.mongodb.org/browse/SERVER-45916</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Consider the following sequence,&lt;br/&gt;
1) Start an index build on collection A on primary.&lt;br/&gt;
2) Prepare the transaction on collection A.&lt;br/&gt;
3) Index build gets aborted can be possibly due to some killOp cmd or due to some key constraint errors.&lt;br/&gt;
4) As a result of index build failure, it tries to do the  cleanup phase. Assume, it&apos;s &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c5cc18dd7484867d82959fc221eeb42efae94255/src/mongo/db/index_builds_coordinator.cpp#L1471&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. So, index build thread has acquired RSTL in mode IX  and the uninterruptible lock guard is enabled.&lt;br/&gt;
5) Now,  assumed stepDown cmd comes in. So, it&apos;s going to enqueue  the RSTL in mode X. But, blocked behind the index build thread.&lt;br/&gt;
6) Now, the index builder thread tries to acquire &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c5cc18dd7484867d82959fc221eeb42efae94255/src/mongo/db/index_builds_coordinator.cpp#L1516&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;collection lock in X mode&lt;/a&gt; to write the abortIndexBuild oplog entry and to tear down the index build. But this step, gets blocked behind prepared transaction due to collection lock conflict.&lt;br/&gt;
7) Prepared transaction&apos;s commit command blocks behind the step down thread.&lt;/p&gt;

</description>
                <environment></environment>
        <key id="1124812">SERVER-45916</key>
            <summary>On primary, 2-phase index build cleanup writes an abortIndexBuild oplog entry under a stronger mode user collection lock X which can lead to 3 way deadlock with prepared transactions, step down and index build</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="louis.williams@mongodb.com">Louis Williams</assignee>
                                    <reporter username="suganthi.mani@mongodb.com">Suganthi Mani</reporter>
                        <labels>
                    </labels>
                <created>Fri, 31 Jan 2020 20:22:46 +0000</created>
                <updated>Sun, 29 Oct 2023 22:12:39 +0000</updated>
                            <resolved>Fri, 17 Apr 2020 17:37:20 +0000</resolved>
                                                    <fixVersion>4.7.0</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3045111" author="xgen-internal-githook" created="Fri, 17 Apr 2020 17:34:02 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Louis Williams&apos;, &apos;email&apos;: &apos;louis.williams@mongodb.com&apos;, &apos;username&apos;: &apos;louiswilliams&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-45916&quot; title=&quot;On primary, 2-phase index build cleanup writes an abortIndexBuild oplog entry under a stronger mode user collection lock X which can lead to 3 way deadlock with prepared transactions, step down and index build&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-45916&quot;&gt;&lt;del&gt;SERVER-45916&lt;/del&gt;&lt;/a&gt; Index build abort should be interruptible by stepDown&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/65c979794dc643c0b432be89de47bda7c8e7bd8c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/65c979794dc643c0b432be89de47bda7c8e7bd8c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3043258" author="louis.williams" created="Thu, 16 Apr 2020 19:58:34 +0000"  >&lt;p&gt;Code review: &lt;a href=&quot;https://mongodbcr.appspot.com/575110005/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://mongodbcr.appspot.com/575110005/&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2778089" author="suganthi.mani" created="Fri, 31 Jan 2020 23:44:16 +0000"  >&lt;p&gt;I would expect something like this, if a old primary index build gets aborted  for some reason  except due to killop cmd, then the newly elected primary also will also abort due to same rason and send an abortIndexBuild oplog entry.&lt;/p&gt;

&lt;p&gt;If that&apos;s the case, then, when the killop cmd was successfully able to interrupt  the parent createIndex thread, the parent createIndex thread after &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c5cc18dd7484867d82959fc221eeb42efae94255/src/mongo/db/commands/create_indexes.cpp#L932&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;aborting&lt;/a&gt; the index build coordinator thread (i.e., changing the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c5cc18dd7484867d82959fc221eeb42efae94255/src/mongo/db/index_builds_coordinator.cpp#L685.&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;aborted&lt;/a&gt; field to true), parent thread should generate the abortIndex oplog entry. To make it work correctly we should also make sure the parent thread holds RSTL in mode IX once we have started the index build. (We are already guaranteeing that by holding the RSTL &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c5cc18dd7484867d82959fc221eeb42efae94255/src/mongo/db/commands/create_indexes.cpp#L891-L892&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;

&lt;p&gt;For other index build abortion due to some erroneous data records, we can behave like we do it for commit index build. The newly elected primary take care of abort and generate abortIndex oplog entry. &lt;/p&gt;</comment>
                            <comment id="2778041" author="suganthi.mani" created="Fri, 31 Jan 2020 22:57:41 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; response for this problem in the google doc&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Ah - I see. The solution is: at stepdown time, we must cancel any active index-abort operations. The killop cmd is ignored. The index build remains active on all nodes, and the new primary continues building the index.&lt;/p&gt;

&lt;p&gt;This applies to any index-commit operations as well (they take the same locks as index-abort). The commit operation is cancelled at stepdown time, and the new primary continues the index build, which will probably be to immediately commit.&lt;/p&gt;&lt;/blockquote&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="1215254">SERVER-46560</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1125152">SERVER-45921</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 7 Feb 2020 20:14:56 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 42 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-46560'>SERVER-46560</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 42 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>suganthi.mani@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwmvh3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr652f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3633">Execution Team 2020-04-06</customfieldvalue>
    <customfieldvalue id="3634">Execution Team 2020-04-20</customfieldvalue>
    <customfieldvalue id="3635">Execution Team 2020-05-04</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwmhqf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>