<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:45:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37199] Yield locks of transactions in secondary application</title>
                <link>https://jira.mongodb.org/browse/SERVER-37199</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Secondary application tends to acquire locks conservatively, for example, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/5603a0812eb64c8009a514a3c59182f7f7134b9e/src/mongo/db/repl/sync_tail.cpp#L308&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;all commands acquire global write lock&lt;/a&gt;. This will conflict with prepared transactions.&#160;We can yield locks of transactions on secondary since the oplog should include no conflicting operations due to the concurrency control on primary.&#160;&lt;/p&gt;

&lt;p&gt;An alternative solution is to&#160;have secondaries acquire the same locks as the primary, but yielding locks will also fix other issues, e.g. &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38121&quot; title=&quot;multikey index ops in a transaction can cause secondaries to hang&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38121&quot;&gt;&lt;del&gt;SERVER-38121&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="606934">SERVER-37199</key>
            <summary>Yield locks of transactions in secondary application</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="siyuan.zhou@mongodb.com">Siyuan Zhou</assignee>
                                    <reporter username="siyuan.zhou@mongodb.com">Siyuan Zhou</reporter>
                        <labels>
                            <label>prepare_durability</label>
                    </labels>
                <created>Wed, 19 Sep 2018 06:16:17 +0000</created>
                <updated>Sun, 29 Oct 2023 22:28:05 +0000</updated>
                            <resolved>Mon, 3 Dec 2018 23:23:45 +0000</resolved>
                                                    <fixVersion>4.1.6</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>14</watches>
                                                                                                                <comments>
                            <comment id="2079335" author="xgen-internal-githook" created="Mon, 3 Dec 2018 23:21:48 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Siyuan Zhou&apos;, &apos;email&apos;: &apos;siyuan.zhou@mongodb.com&apos;, &apos;username&apos;: &apos;visualzhou&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37199&quot; title=&quot;Yield locks of transactions in secondary application&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37199&quot;&gt;&lt;del&gt;SERVER-37199&lt;/del&gt;&lt;/a&gt; Yield locks of transactions in secondary application.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/55e72b015e2aa7297c00db29e4d93451ea61a7ca&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/55e72b015e2aa7297c00db29e4d93451ea61a7ca&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2063741" author="judah.schvimer" created="Thu, 15 Nov 2018 20:38:09 +0000"  >&lt;p&gt;I would rather yield locks and reacquire them. It makes me less nervous about accidentally allowing in readers without generating prepare conflicts. That said, I agree solution (1) would work. I don&apos;t follow how parallel application would be made more difficult by yielding locks though.&lt;/p&gt;</comment>
                            <comment id="2062595" author="siyuan.zhou@10gen.com" created="Thu, 15 Nov 2018 01:02:15 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, I see the point of recovering locks for step-up. There&#8217;re two solutions: 1) drop locks for prepared transaction on secondary, then abort and reapply them on step-up. 2) yield locks for prepared transactions on secondary, then resume the locks on step-up. The first one is easier on secondary application, but harder on step-up. The second solution has more work to do on secondary application, but less on step-up. When we have beyond 16MB transactions, yielding and restoring locks will be needed for all transactional operations in solution #2, while solution #1 makes things simpler by dropping the locks.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; and I thought solution #1 was easier without considering the state transition. Now I&apos;m leaning towards solution #2 due to performance concerns of solution #1 on step-up and its complexity of reapplication.&lt;/p&gt;

&lt;p&gt;However, if secondary application is going to differ from primary&apos;s behavior further when we apply transactions in parallel (e.g they stash ops in different ways), it might be more straightforward to re-apply the ops rather than recovering them on state transitions.&lt;/p&gt;</comment>
                            <comment id="2062479" author="judah.schvimer" created="Wed, 14 Nov 2018 22:37:55 +0000"  >&lt;p&gt;Prepared transactions will need to recover their locks on step up while the RSTL is held. To make step-up writes (mainly dropping temporary collections) not conflict with prepared transactions, we should recover the prepared transaction locks at the very end of the step up, right before releasing the RSTL.&lt;/p&gt;</comment>
                            <comment id="2053891" author="geert.bosch" created="Tue, 6 Nov 2018 21:54:01 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;, I confirmed for &lt;tt&gt;listCollections&lt;/tt&gt;. We still have a few commands that take &lt;tt&gt;MODE_S&lt;/tt&gt; locks, such as &lt;tt&gt;dbStats&lt;/tt&gt; and &lt;tt&gt;dbHash&lt;/tt&gt; as Tess mentions. We&apos;re planning to get rid of these as well.&lt;/p&gt;</comment>
                            <comment id="2053790" author="tess.avitabile" created="Tue, 6 Nov 2018 20:57:26 +0000"  >&lt;p&gt;We still use S mode for dbhash&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/ce846fd85d7624504a0508706a1baf099817741f/src/mongo/db/commands/dbhash.cpp#L119&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. However, I don&apos;t think it&apos;s important that prepared transactions conflict with dbhash on secondaries.&lt;/p&gt;</comment>
                            <comment id="2053781" author="siyuan.zhou@10gen.com" created="Tue, 6 Nov 2018 20:52:41 +0000"  >&lt;p&gt;Discussed an alternative solution with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;: we may yield locks after transactions get prepared on secondaries, so that other commands don&apos;t conflict with them, because the concurrency control on primary guarantees there&apos;s no conflict if the ops are applied in the oplog order. One concern from Judah was the behavioral change of operations that hold locks in S mode on secondaries, which conflict with prepared transactions&apos; IX locks. According to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;, we&apos;ve removed all S mode locks on master. For example, &lt;tt&gt;listCollections&lt;/tt&gt; needs a DB lock in IS mode and then each collection lock in IS mode, rather than a DB lock in S mode. However,&#160; to support the yielding behavior, we probably need to change WriteUnitOfWork to introduce a new &lt;tt&gt;prepared&lt;/tt&gt; state for a recovery unit.&#160;WriteUnitOfWork is a RAII type designed to represent the two-phase locking, yielding violates that semantics.&lt;/p&gt;

&lt;p&gt;That being said, we&apos;ll keep investigating the original solution. After all, it makes the system easier to reason about if operations on secondaries acquire the same locks as on the primary.&lt;/p&gt;</comment>
                            <comment id="2007998" author="siyuan.zhou@10gen.com" created="Wed, 19 Sep 2018 16:59:39 +0000"  >&lt;p&gt;We should also audit the applyOps command&apos;s oplog entry path to make sure there&apos;s no ways to generate oplog entries that lock Global during oplog application.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="639092">SERVER-38282</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="633473">SERVER-38121</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="652186">SERVER-38588</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="610532">SERVER-37336</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="683624">SERVER-39372</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="741394">SERVER-40723</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="685557">SERVER-39424</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="630322">SERVER-37988</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="609716">SERVER-37313</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 6 Nov 2018 20:57:26 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 10 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1032</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 10 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>62.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu8d8v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htyten:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2541">Repl 2018-10-08</customfieldvalue>
    <customfieldvalue id="2542">Repl 2018-10-22</customfieldvalue>
    <customfieldvalue id="2543">Repl 2018-11-05</customfieldvalue>
    <customfieldvalue id="2605">Repl 2018-11-19</customfieldvalue>
    <customfieldvalue id="2606">Repl 2018-12-03</customfieldvalue>
    <customfieldvalue id="2607">Repl 2018-12-17</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu7zi7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>