<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:07:13 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-44870] MongoDB to allow WiredTiger to return WT_ROLLBACK for all transactions</title>
                <link>https://jira.mongodb.org/browse/SERVER-44870</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently, WT in MongoDB will not return WT_ROLLBACK for transactions that have not yet performed an update. These cases could be desirable for system stability.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1027888">SERVER-44870</key>
            <summary>MongoDB to allow WiredTiger to return WT_ROLLBACK for all transactions</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="milkie@mongodb.com">Eric Milkie</reporter>
                        <labels>
                    </labels>
                <created>Wed, 27 Nov 2019 17:08:33 +0000</created>
                <updated>Tue, 21 Nov 2023 17:45:50 +0000</updated>
                                                                            <component>Storage</component>
                                        <votes>1</votes>
                                    <watches>22</watches>
                                                                                                                <comments>
                            <comment id="4359187" author="keith.bostic" created="Wed, 16 Feb 2022 17:08:48 +0000"  >&lt;p&gt;FWIW, this is still an issue: BF-24076/&lt;a href=&quot;https://jira.mongodb.org/browse/WT-8713&quot; title=&quot;Cache stuck with metadata updates&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-8713&quot;&gt;&lt;del&gt;WT-8713&lt;/del&gt;&lt;/a&gt; is a ticket where an older reader appears to have caused the cache to time out and fail the system, and having the ability to fail that specific transaction would have prevented the failure.&lt;/p&gt;</comment>
                            <comment id="3298096" author="brian.lane" created="Thu, 23 Jul 2020 06:02:20 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=connie.chen&quot; class=&quot;user-hover&quot; rel=&quot;connie.chen&quot;&gt;connie.chen&lt;/a&gt; - that&apos;s fine.   Could this perhaps be tagged as a candidate for a QW in a future quarter?&lt;/p&gt;</comment>
                            <comment id="3293056" author="connie.chen" created="Mon, 20 Jul 2020 18:33:41 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=brian.lane&quot; class=&quot;user-hover&quot; rel=&quot;brian.lane&quot;&gt;brian.lane&lt;/a&gt;&#160;- is it ok to still leave this in backlog for now? Has this become more urgent?&#160;&lt;/p&gt;</comment>
                            <comment id="2655230" author="brian.lane" created="Fri, 20 Dec 2019 05:45:02 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=keith.bostic&quot; class=&quot;user-hover&quot; rel=&quot;keith.bostic&quot;&gt;keith.bostic&lt;/a&gt;, I will put this in the backlog for now.&lt;/p&gt;</comment>
                            <comment id="2634462" author="keith.bostic" created="Mon, 16 Dec 2019 17:04:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, based on &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.gorrod&quot; class=&quot;user-hover&quot; rel=&quot;alexander.gorrod&quot;&gt;alexander.gorrod&lt;/a&gt;&apos;s comments in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44870&quot; title=&quot;MongoDB to allow WiredTiger to return WT_ROLLBACK for all transactions&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44870&quot;&gt;SERVER-44870&lt;/a&gt;, I&apos;ve updated the ticket to broaden its scope: as I now understand it, the storage layer won&apos;t return WT_ROLLBACK for any MongoDB transaction that has yet to do an update.&lt;/p&gt;

&lt;p&gt;I can&apos;t speak to how important this is to server operations, but where we see this problem in our test programs is a thread with a read timestamp in the past that hasn&apos;t yet done an update, and which has been grabbed to help with eviction. When eviction is having trouble and the read timestamp is a problem, pinning the reader inside the storage eviction loop can just make things worse, of course.&lt;/p&gt;

&lt;p&gt;Given the broader scope of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44870&quot; title=&quot;MongoDB to allow WiredTiger to return WT_ROLLBACK for all transactions&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44870&quot;&gt;SERVER-44870&lt;/a&gt;, perhaps we should revisit its scheduling (or decide it&apos;s more work than originally understood and should wait for a later release).&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2597981" author="alexander.gorrod" created="Wed, 11 Dec 2019 00:11:06 +0000"  >&lt;blockquote&gt;&lt;p&gt;What I believe to be the case is that once a transaction does something that gets it allocated a transaction ID, then the WiredTiger layer will return WT_ROLLBACK, expecting the caller is able to handle it.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I agree - that&apos;s how it works. Note that I pulled something out of the middle of the excerpt there.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;If it&apos;s an implicit transaction, that has yet to do an update/insert, then the WiredTiger layer won&apos;t return WT_ROLLBACK, because we&apos;re not sure the MongoDB operation can handle it.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;That&apos;s true too, but it&apos;s not only applicable to implicit transactions.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;... an update/insert, or, being part of an explicit transaction created in the application&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;All transactions avoid allocating a transaction ID internally until they do an update. When an explicit transaction is begun it will allocate a snapshot if necessary, but it doesn&apos;t get a transaction ID until it needs one (which is when it goes to do an update operation).&lt;/p&gt;</comment>
                            <comment id="2587046" author="louis.williams" created="Thu, 5 Dec 2019 22:48:53 +0000"  >&lt;p&gt;We already have a ticket to assert that normal cursor operations use explicit transactions, because we believe they should: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44582&quot; title=&quot;Assert that a storage transaction is active for all cursor read operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44582&quot;&gt;&lt;del&gt;SERVER-44582&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I think we should use this ticket to evaluate whether the current users of &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed03960eb32b62f2f62df7eafe08852143283269/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.h#L188&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;getSessionNoTxn&lt;/a&gt; can handle WT_ROLLBACK errors.&lt;/p&gt;</comment>
                            <comment id="2586982" author="keith.bostic" created="Thu, 5 Dec 2019 22:18:11 +0000"  >&lt;p&gt;In talking with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;, I think I&apos;ve spotted the problem.&lt;/p&gt;

&lt;p&gt;I&apos;ve not been clear that I&apos;m talking about implicit transactions. What I believe to be the case is that once a transaction does something that gets it allocated a transaction ID (so, an update/insert, or, being part of an explicit transaction created in the application), then the WiredTiger layer will return WT_ROLLBACK, expecting the caller is able to handle it. If it&apos;s an implicit transaction, that has yet to do an update/insert, then the WiredTiger layer won&apos;t return WT_ROLLBACK, because we&apos;re not sure the MongoDB operation can handle it.&lt;/p&gt;

&lt;p&gt;This isn&apos;t really my area of expertise, so YMMV, but I think that&apos;s how it works.&lt;/p&gt;</comment>
                            <comment id="2586495" author="keith.bostic" created="Thu, 5 Dec 2019 18:14:13 +0000"  >&lt;blockquote&gt;
&lt;p&gt;I think we&apos;re using different terms for &quot;read-only&quot;. You&apos;re talking about WT &quot;readonly=true&quot; cursors, correct? We don&apos;t use those in MongoDB outside --queryableBackupMode. When we talk about reads in MongoDB, I mean that we start a transaction and abort it at the end of the operation without doing any writes.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;No, I&apos;m talking about cursors that have not (yet) performed any update operations.&lt;/p&gt;</comment>
                            <comment id="2586128" author="louis.williams" created="Thu, 5 Dec 2019 16:34:54 +0000"  >&lt;p&gt;I think we&apos;re using different terms for &quot;read-only&quot;. You&apos;re talking about WT &quot;readonly=true&quot; cursors, correct? We don&apos;t use those in MongoDB outside --queryableBackupMode. When we talk about reads in MongoDB, I mean that we start a transaction and abort it at the end of the operation without doing any writes. We don&apos;t pass any special configuration to WT. In this case, WT_ROLLBACK can be returned, correct? Is there a plan to start using read-only transactions in MongoDB?&lt;/p&gt;

&lt;p&gt;For all operations, we convert WT_ROLLBACK to a WriteConflictException. In MongoDB, outside of a writeConflictRetry(), read operations will pass this error to clients. Internal threads are definitely not prepared to handle WT_ROLLBACK for reads, but I imagine this is already a problem today.&lt;/p&gt;</comment>
                            <comment id="2585906" author="keith.bostic" created="Thu, 5 Dec 2019 15:13:30 +0000"  >&lt;blockquote&gt;
&lt;p&gt;My understanding is that WT already returns WT_ROLLBACK for reads when the cache is full.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I don&apos;t believe that&apos;s true.&lt;/p&gt;

&lt;p&gt;The code that raised this question is when an application thread is tapped to help with eviction. In that case, if the transaction is readonly, it will not return WT_ROLLBACK (or any error), even if the transaction is contributing to the problem (for example, it&apos;s an older reading pinning content in the cache). I haven&apos;t done a review, but in general I think it&apos;s the case that the places where WiredTiger returns WT_ROLLBACK are not cases triggered by readonly operations.&lt;/p&gt;

&lt;p&gt;I don&apos;t know any of the MongoDB history on this, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.gorrod&quot; class=&quot;user-hover&quot; rel=&quot;alexander.gorrod&quot;&gt;alexander.gorrod&lt;/a&gt; just told me the upper levels of MongoDB weren&apos;t prepared to handle WT_ROLLBACK from readonly operations.&lt;/p&gt;</comment>
                            <comment id="2585880" author="louis.williams" created="Thu, 5 Dec 2019 15:02:14 +0000"  >&lt;p&gt;My understanding is that WT already returns WT_ROLLBACK for reads when the cache is full. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=keith.bostic&quot; class=&quot;user-hover&quot; rel=&quot;keith.bostic&quot;&gt;keith.bostic&lt;/a&gt;, Eric said you might be able to elaborate on how&#160;this is different than what we do today?&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1026825">WT-5266</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2498588">SERVER-83186</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1060167">WT-5330</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1988392">WT-8869</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5006R00001mDn1MQAS, 5006R00001mdAiMQAU, 5006R00001rYfH3QAK]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 5 Dec 2019 15:02:14 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 51 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 51 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexander.gorrod@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>brian.lane@mongodb.com</customfieldvalue>
            <customfieldvalue>connie.chen@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>keith.bostic@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw6naf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr24kv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw69jr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>