<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:59:34 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-64125] MDB server sets the commit/durable timestamps equal to the stable timestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-64125</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Generally, it is not correct to allow a transaction to commit with a commit timestamp at the stable timestamp, or a prepared transaction to commit with a durable timestamp at the stable timestamp. This can confuse checkpoint as to whether the newly committed transaction should be included in the checkpoint and can potentially lead to data inconsistencies.&lt;/p&gt;

&lt;p&gt;With the merge of &lt;a href=&quot;https://jira.mongodb.org/browse/WT-7712&quot; title=&quot;commit and durable timestamps should be disallowed at stable timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-7712&quot;&gt;&lt;del&gt;WT-7712&lt;/del&gt;&lt;/a&gt;, WiredTiger standalone builds fail attempts to set the commit or durable timestamps to the stable timestamp, but MDB Server builds are allowed to do so.&lt;/p&gt;

&lt;p&gt;We do not know of any actual MDB Server problems in this area, but it would be good to fix any place where this happens and change the WiredTiger standalone behavior apply to all builds, to avoid the introduction of problems in the future.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1993272">SERVER-64125</key>
            <summary>MDB server sets the commit/durable timestamps equal to the stable timestamp</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="jordi.olivares-provencio@mongodb.com">Jordi Olivares Provencio</assignee>
                                    <reporter username="keith.bostic@mongodb.com">Keith Bostic</reporter>
                        <labels>
                    </labels>
                <created>Wed, 2 Mar 2022 19:26:11 +0000</created>
                <updated>Sun, 29 Oct 2023 21:41:44 +0000</updated>
                            <resolved>Fri, 1 Apr 2022 10:04:03 +0000</resolved>
                                                    <fixVersion>6.0.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>13</watches>
                                                                                                                <comments>
                            <comment id="4449840" author="xgen-internal-githook" created="Fri, 1 Apr 2022 09:58:48 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Jordi Olivares Provencio&apos;, &apos;email&apos;: &apos;jordi.olivares-provencio@mongodb.com&apos;, &apos;username&apos;: &apos;jordiolivares&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-64125&quot; title=&quot;MDB server sets the commit/durable timestamps equal to the stable timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-64125&quot;&gt;&lt;del&gt;SERVER-64125&lt;/del&gt;&lt;/a&gt; Avoid committing at the stable timestamp&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/c2de464c661ec8fc3198373932d0e3aab7fde971&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/c2de464c661ec8fc3198373932d0e3aab7fde971&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4410224" author="daniel.gottlieb@10gen.com" created="Mon, 14 Mar 2022 15:26:00 +0000"  >&lt;p&gt;Thanks for running that experiment again Keith.&lt;/p&gt;

&lt;p&gt;I took a look. I only counted 3 unique failures, so I&apos;ll list them here:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;One is a disk_wiredtiger repair test. We timestamp writes when writing to the oplog (for historical reasons in 3.6 to make sure oplog entries were correctly made visible for a &lt;tt&gt;read_timestamp&lt;/tt&gt;). We can remove that now as all oplog reads use MDB logic for visibility. Not sure if that&apos;s the only fallout in this test as repair/recovery are the cases I&apos;d expect to see trip this assertion.&lt;/li&gt;
	&lt;li&gt;storage_timestamp_tests. Same timestamp call when writing to the oplog. Being a unittest, I&apos;m not worried about this being a blocker.&lt;/li&gt;
	&lt;li&gt;The only real complaint was when we reconstruct prepared transactions that flip multikey. More detail:&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;A disclaimer, everything about how multikey is tracked is considered a wart. So while describing the behavior, please don&apos;t believe there&apos;s some virtue to how we do things. We got here because of constraints dating back to WT&apos;s adoption and haven&apos;t made the effort to schema our way out of this problem as it&apos;s a tech debt problem that carries on-disk format changes (i.e: upgrade/downgrade) and large risk.&lt;/p&gt;

&lt;p&gt;Multikey is an index state when (most simply) an index contains multiple keys to the same MDB document/record. When an index is not* multikey, query can assume any document returned from an index is unique. But if an index is multikey, query must maintain a set of returned records to avoid double-returning the same document.&lt;/p&gt;

&lt;p&gt;When a client inserts the first document that &quot;flips multikey&quot; we do that write on the _mdb_catalog document. Unfortunately that document tracks a bunch of detail, not limited to the specific index. Meaning preparing a change to that document can cause contention for things that are logically unrelated. Thus for things like flipping multikey in a prepared transaction, we make an effort to change that document in a separate transaction and commit it prior to the data write that it reflects. The system remains correct so long as multikey is true at or before there are multikey documents.&lt;/p&gt;

&lt;p&gt;When reconstructing prepared transactions, we can find that multikey needs to be set (I&apos;m not entirely sure why this happens for txns in a prepare state at the stable timestamp &amp;#8211; understanding that may provide us with more outs of this situation). Right now we when reconstructing we can &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e1ab3976f52497cb516b95c752eb81d4a69e3bbf/src/mongo/db/catalog/index_catalog_entry_impl.cpp#L315-L326&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;write at the stable timestamp&lt;/a&gt;. I think it&apos;s just as safe here to write at stable + 1.&lt;/p&gt;

&lt;p&gt;That said, I do have a non-sequitur concern about these recovery writes where the multikey write has a timestamp larger than the prepare timestamp. It means that after the transaction commits, the transaction could choose a visible (commit_)timestamp smaller than the multikey write. This isn&apos;t strictly a problem today where multikey is read from a modern in-memory state (multikey doesn&apos;t &quot;go backwards&quot;). But this would be wrong in a versioned catalog world where multikey state is derived from the reader&apos;s snapshot.&lt;/p&gt;</comment>
                            <comment id="4407872" author="keith.bostic" created="Fri, 11 Mar 2022 22:20:36 +0000"  >&lt;p&gt;Now that &lt;a href=&quot;https://jira.mongodb.org/browse/WT-7712&quot; title=&quot;commit and durable timestamps should be disallowed at stable timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-7712&quot;&gt;&lt;del&gt;WT-7712&lt;/del&gt;&lt;/a&gt; has been merged into master (and a few other 5.3 inspired odds and ends have settled down), I wanted to rerun the &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-64125&quot; title=&quot;MDB server sets the commit/durable timestamps equal to the stable timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-64125&quot;&gt;&lt;del&gt;SERVER-64125&lt;/del&gt;&lt;/a&gt; patch test. It looks like there are roughly 9 unique test failures caused by updating WiredTiger to reject commits equal to the stable timestamp (assuming tests failing on different platforms are failing in the same way).&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://spruce.mongodb.com/version/622ba9f932f41735e6af12a8/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://spruce.mongodb.com/version/622ba9f932f41735e6af12a8/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;cc: &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.gorrod&quot; class=&quot;user-hover&quot; rel=&quot;alexander.gorrod&quot;&gt;alexander.gorrod&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4387522" author="keith.bostic" created="Wed, 2 Mar 2022 20:07:28 +0000"  >&lt;p&gt;The specific changes to reverse are &lt;a href=&quot;https://github.com/wiredtiger/wiredtiger/blob/1c6a4c6990c30621dda6d84a773c5ce34c5ef1ae/src/txn/txn_timestamp.c#L566-L576&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;&#160;and &lt;a href=&quot;https://github.com/wiredtiger/wiredtiger/blob/1c6a4c6990c30621dda6d84a773c5ce34c5ef1ae/src/txn/txn_timestamp.c#L675-L685&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="1792212">WT-7712</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1993289">WT-8902</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="2038247">SERVER-66139</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 14 Mar 2022 15:26:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 44 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/WT-7712'>WT-7712</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 44 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>35.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jordi.olivares-provencio@mongodb.com</customfieldvalue>
            <customfieldvalue>keith.bostic@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0luwn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i054kg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5812">Execution Team 2022-03-21</customfieldvalue>
    <customfieldvalue id="5813">Execution Team 2022-04-04</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0lh1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>