<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:45:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37336] Test that background index build do not block on prepared transactions on secondaries</title>
                <link>https://jira.mongodb.org/browse/SERVER-37336</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Background index build on secondaries acquires a database lock in X mode and releases it at the beginning, scans the collection with IX lock, and acquires an X lock at the end. The first X lock is acquired by &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9be782c954e7d24192c08252b98fe58e52c2071d/src/mongo/db/index_builder.cpp#L224&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the background job&lt;/a&gt; but the running command &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9be782c954e7d24192c08252b98fe58e52c2071d/src/mongo/db/repl/oplog.cpp#L267&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;waits for it&lt;/a&gt;, so it won&apos;t conflict with transaction operations. The IX lock plays well with transactions. However, the final X lock will cause problem on secondaries.&lt;/p&gt;

&lt;p&gt;If the background index build completion is blocked by a prepared transaction and replication application on the same database is blocked by the background index build completion. That&apos;s a deadlock. The problem is similar to the 2-phase drop locking issue we&apos;ve seen in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34349&quot; title=&quot;Wait for drop to be majority committed in transaction tests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34349&quot;&gt;&lt;del&gt;SERVER-34349&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="610532">SERVER-37336</key>
            <summary>Test that background index build do not block on prepared transactions on secondaries</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="backlog-server-repl">Backlog - Replication Team</assignee>
                                    <reporter username="siyuan.zhou@mongodb.com">Siyuan Zhou</reporter>
                        <labels>
                            <label>prepare_optional</label>
                            <label>prepare_testing</label>
                    </labels>
                <created>Wed, 26 Sep 2018 23:59:07 +0000</created>
                <updated>Fri, 27 Oct 2023 20:43:15 +0000</updated>
                            <resolved>Fri, 14 Dec 2018 00:10:34 +0000</resolved>
                                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="2090540" author="judah.schvimer" created="Fri, 14 Dec 2018 00:10:19 +0000"  >&lt;p&gt;Closing per &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;&apos;s comment that any work and testing will be done as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38588&quot; title=&quot;Hybrid index builds do not work when applied concurrently with prepared transactions on secondaries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38588&quot;&gt;&lt;del&gt;SERVER-38588&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2090256" author="siyuan.zhou@10gen.com" created="Thu, 13 Dec 2018 20:45:51 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, we discovered this issue by code inspection. Discussed offline with storage team as summarized in this comment, we believe &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38588&quot; title=&quot;Hybrid index builds do not work when applied concurrently with prepared transactions on secondaries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38588&quot;&gt;&lt;del&gt;SERVER-38588&lt;/del&gt;&lt;/a&gt; can be fixed without blocking background index builds on active prepared transactions. Thus there is no work for this ticket. Because this is a race condition on secondary, we would have to use fail points to coordinate the events in testing. Given that hybrid index builds will essentially make all index builds background, we will have a pretty good test coverage for both primary and secondary. I think a specific test for this has a low priority.&lt;/p&gt;</comment>
                            <comment id="2089735" author="louis.williams" created="Thu, 13 Dec 2018 16:29:35 +0000"  >&lt;p&gt;There is a current issue with how hybrid, background indexes behave with prepared transactions, summarized in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38588&quot; title=&quot;Hybrid index builds do not work when applied concurrently with prepared transactions on secondaries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38588&quot;&gt;&lt;del&gt;SERVER-38588&lt;/del&gt;&lt;/a&gt;, that makes me believe background index builds &lt;b&gt;should&lt;/b&gt; block on active prepared transactions.&lt;/p&gt;</comment>
                            <comment id="2057430" author="milkie" created="Fri, 9 Nov 2018 18:27:58 +0000"  >&lt;p&gt;We discovered that the try-lock idea isn&apos;t viable.  Going to explore other alternative solutions; one solution is to have prepared transactions not hold any locks when on secondary nodes.&lt;/p&gt;</comment>
                            <comment id="2049688" author="milkie" created="Thu, 1 Nov 2018 22:48:39 +0000"  >&lt;p&gt;I think we could avoid that by enqueuing a lock request for a limited amount of time, rather than the current tryLock behavior of just looking to see what mode the resource is currently locked in.  Would that work?&lt;/p&gt;</comment>
                            <comment id="2049318" author="geert.bosch" created="Thu, 1 Nov 2018 18:57:25 +0000"  >&lt;p&gt;I&apos;m not so much concerned about the tryLock starving IX locks, but more about the opposite: on a busy system, there may always be open transactions and we&apos;ll be trying forever.&lt;/p&gt;</comment>
                            <comment id="2047976" author="siyuan.zhou@10gen.com" created="Wed, 31 Oct 2018 19:24:02 +0000"  >&lt;p&gt;I believe Eric&apos;s solution will work assuming spinning in tryLock X lock doesn&apos;t starve other IX locks.&lt;/p&gt;</comment>
                            <comment id="2029540" author="milkie" created="Wed, 10 Oct 2018 21:00:07 +0000"  >&lt;p&gt;Note that I do not believe it would be possible to require users to somehow cease all background index builds across all nodes of a replica set before running setFCV. We would have to build some new machinery for that, and in addition it would be a pretty onerous upgrade requirement. &lt;/p&gt;</comment>
                            <comment id="2029529" author="milkie" created="Wed, 10 Oct 2018 20:58:05 +0000"  >&lt;p&gt;After some discussion yesterday afternoon, we had an idea: we could change the  index build&#8217;s acquisition of a DB X lock at the end of the build to spin in tryLock, thus avoiding a deadlock with any prepared transactions. &lt;/p&gt;</comment>
                            <comment id="2026881" author="greg.mckeon" created="Mon, 8 Oct 2018 18:07:01 +0000"  >&lt;p&gt;If you have in-progress background index builds using the 4.0 method, you could hit a deadlock once you upgrade to FCV 4.2 if they were still running and you run prepareTransaction. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; or &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dianna.hohensee&quot; class=&quot;user-hover&quot; rel=&quot;dianna.hohensee&quot;&gt;dianna.hohensee&lt;/a&gt;, would you consider not allowing a user to upgrade to FCV 4.2 while a background index build was in progress?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="606934">SERVER-37199</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="652186">SERVER-38588</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 28 Sep 2018 14:03:55 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 8 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/PM-253'>PM-253</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1032</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 8 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-repl</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu8yqv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htzdcn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2595">Storage NYC 2018-11-05</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu8l07:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>