<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:53:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-40013] upgrade downgrade support for config.transactions startTimestamp field</title>
                <link>https://jira.mongodb.org/browse/SERVER-40013</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39680&quot; title=&quot;Maintain the oldest active transaction timestamp only with the transaction table&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39680&quot;&gt;&lt;del&gt;SERVER-39680&lt;/del&gt;&lt;/a&gt; introduced a &lt;tt&gt;startTimestamp&lt;/tt&gt; field. We need to add upgrade/downgrade support similar to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36498&quot; title=&quot;Remove &amp;#39;config.transactions&amp;#39; entries with &amp;#39;state&amp;#39; field on shutdown in fCV 4.0&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36498&quot;&gt;&lt;del&gt;SERVER-36498&lt;/del&gt;&lt;/a&gt; since lower versions won&apos;t recognize it and 4.2 won&apos;t have it on upgrade.&lt;/p&gt;</description>
                <environment></environment>
        <key id="712375">SERVER-40013</key>
            <summary>upgrade downgrade support for config.transactions startTimestamp field</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="jason.chan@mongodb.com">Jason Chan</assignee>
                                    <reporter username="judah.schvimer@mongodb.com">Judah Schvimer</reporter>
                        <labels>
                            <label>bigtxns_upgrade_downgrade</label>
                            <label>todo_in_code</label>
                    </labels>
                <created>Thu, 7 Mar 2019 15:33:08 +0000</created>
                <updated>Wed, 2 Oct 2019 18:45:14 +0000</updated>
                            <resolved>Wed, 17 Apr 2019 20:21:50 +0000</resolved>
                                                                    <component>Replication</component>
                    <component>Upgrade/Downgrade</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="2447928" author="xgen-internal-githook" created="Wed, 2 Oct 2019 18:45:14 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;jasonjhchan&apos;, &apos;email&apos;: &apos;jason.chan@mongodb.com&apos;, &apos;name&apos;: &apos;Jason Chan&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-42540&quot; title=&quot;Complete TODO listed in SERVER-40013&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-42540&quot;&gt;&lt;del&gt;SERVER-42540&lt;/del&gt;&lt;/a&gt; Remove TODO from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40013&quot; title=&quot;upgrade downgrade support for config.transactions startTimestamp field&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-40013&quot;&gt;&lt;del&gt;SERVER-40013&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/27dab1976592948aabfffdfdb98b1f8313bb8741&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/27dab1976592948aabfffdfdb98b1f8313bb8741&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2217046" author="jason.chan" created="Wed, 17 Apr 2019 20:21:50 +0000"  >&lt;p&gt;Closing ticket as there is no work needed after closer evaluation.&lt;/p&gt;</comment>
                            <comment id="2217036" author="siyuan.zhou@10gen.com" created="Wed, 17 Apr 2019 20:17:18 +0000"  >&lt;p&gt;SGTM. Great summary!&lt;/p&gt;</comment>
                            <comment id="2216990" author="judah.schvimer" created="Wed, 17 Apr 2019 19:47:24 +0000"  >&lt;p&gt;Ah that makes sense. SGTM to leave out the invariant.&lt;/p&gt;</comment>
                            <comment id="2216978" author="jason.chan" created="Wed, 17 Apr 2019 19:40:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I accidentally left out another potential reason why the invariant to check only the in-memory transaction state wouldn&apos;t work from my previous comment. 4.0 format transactions have an &lt;tt&gt;inProgress&lt;/tt&gt; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b6d336bee9c7adb334333bcb22c432d376458af3/src/mongo/db/transaction_participant.cpp#L456&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;in-memory state&lt;/a&gt; but no on-disk state so I don&apos;t see a way to verify that there are no longer any in-progress 4.2 format transactions without directly looking at the &lt;tt&gt;config.transactions&lt;/tt&gt; table.&lt;/p&gt;

&lt;p&gt;Siyuan was comfortable with foregoing the invariant if adding one required reformatting/reworking parts of the code that might make it less intuitive to the readers. I don&apos;t see a convenient place to stick the invariant without rewriting parts of &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf774fb0a5e9bed81bc475585deb70812a2c9c41/src/mongo/db/commands/set_feature_compatibility_version_command.cpp#L97&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;forEachSessionWithCheckout&lt;/a&gt; and doing a disk look-up directly. Do you have any thoughts?&lt;/p&gt;</comment>
                            <comment id="2216919" author="judah.schvimer" created="Wed, 17 Apr 2019 18:59:55 +0000"  >&lt;p&gt;That all makes sense!&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;validation function here only checks for the in-memory state&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;The in-memory state should reflect the on-disk state or be refreshed from disk implicitly on session checkout. I think adding such an invariant, especially on such a rare path would be valuable.&lt;/p&gt;</comment>
                            <comment id="2216496" author="jason.chan" created="Wed, 17 Apr 2019 14:46:16 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The intention of this ticket was to ensure the consistency of config.transactions after upgrade/downgrade regarding the startOpTime but after discussing with Siyuan, I believe there may actually be no work at all required for this ticket.&lt;/p&gt;

&lt;p&gt;On upgrade: No work needed to be done as all 4.0 transactions in config.transactions would have been committed and therefore need no startOpTime on upgrade. Committed transactions do not have a startOpTime field as they are no longer active.&lt;/p&gt;

&lt;p&gt;On downgrade: After talking to Siyuan, we convinced ourselves that it should be impossible for a 4.2 format transaction to have an &lt;tt&gt;inProgress&lt;/tt&gt; state field in &lt;tt&gt;config.transactions&lt;/tt&gt; when we are downgrading the transaction table. We grab and release the global lock in S mode &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6fd5b4d17b87aa652381fd28cefe2a7eb5ec8d5d/src/mongo/db/commands/set_feature_compatibility_version_command.cpp#L375&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here &lt;/a&gt; to ensure that we wait for all running transactions to have been committed/aborted before we downgrade the transaction table. Once the lock is released, only 4.0 format transactions are allowed as we gate the new oplog format on FCV &amp;lt; fullyUpgradedTo42. Therefore, I believe there is no work needed for this case either.&lt;/p&gt;

&lt;p&gt;I originally wanted to add an invariant in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed0939a343ac78527e2633301b68f52721f93d0a/src/mongo/db/commands/set_feature_compatibility_version_command.cpp#L162-L163&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;downgradeTransactionTable&lt;/a&gt; as a sanity check to make sure no transactions are in the &lt;tt&gt;inProgress&lt;/tt&gt; state, but the validation function here only checks for the in-memory state. I am willing to forego adding the invariant as to avoid the extra disk look-up in the validation step of &lt;tt&gt;forEachSessionWithCheckout&lt;/tt&gt;.&lt;/p&gt;</comment>
                            <comment id="2209966" author="judah.schvimer" created="Thu, 11 Apr 2019 17:33:09 +0000"  >&lt;p&gt;SGTM.&lt;/p&gt;</comment>
                            <comment id="2209887" author="siyuan.zhou@10gen.com" created="Thu, 11 Apr 2019 16:54:48 +0000"  >&lt;p&gt;SGTM. When reading the code, my only question is about concurrency. We don&#8217;t check out the sessions, implying the concurrency isn&#8217;t a concern here. If that&#8217;s true, why do we need to double check its state?&lt;/p&gt;</comment>
                            <comment id="2209815" author="jason.chan" created="Thu, 11 Apr 2019 16:13:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I had some questions regarding the scope/work around this ticket. Judah pointed out that the problem with upgrade/downgrade with regards to the &lt;tt&gt;startOpTime&lt;/tt&gt; (it&apos;s been changed from Timestamp to OpTime) is that on downgrade, lower versions won&apos;t recognize the &lt;tt&gt;startOpTime&lt;/tt&gt; field and on upgrade to 4.2, the field won&apos;t exist in the table.&lt;/p&gt;

&lt;p&gt;After investigating &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36498&quot; title=&quot;Remove &amp;#39;config.transactions&amp;#39; entries with &amp;#39;state&amp;#39; field on shutdown in fCV 4.0&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36498&quot;&gt;&lt;del&gt;SERVER-36498&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39680&quot; title=&quot;Maintain the oldest active transaction timestamp only with the transaction table&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39680&quot;&gt;&lt;del&gt;SERVER-39680&lt;/del&gt;&lt;/a&gt; I wanted to clarify some observations that I made:&lt;/p&gt;

&lt;p&gt;1. In 4.0, only committed transactions will have records in the transaction table. With the current behavior from &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jesse&quot; class=&quot;user-hover&quot; rel=&quot;jesse&quot;&gt;jesse&lt;/a&gt;&apos;s ticket, committed entries do not have a startOpTime field as the value is no longer necessary as the transaction is no longer active. This leads me to believe that no work may be necessary in the upgrade case.&lt;/p&gt;

&lt;p&gt;2. Currently on downgrade to 4.0, all the transaction entries with a &apos;state&apos; field will be removed from the &lt;tt&gt;config.transactions&lt;/tt&gt; table. The only work here should be to simply extend the list of states &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0be76c660815bfdf898dec790905eab6f0b70f0a/src/mongo/db/commands/set_feature_compatibility_version_command.cpp#L163&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; to include the &lt;tt&gt;inTxn&lt;/tt&gt; state.&lt;/p&gt;

&lt;p&gt;Does this implementation plan sound correct to you guys?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10720">
                    <name>Cloners</name>
                                            <outwardlinks description="clones">
                                        <issuelink>
            <issuekey id="705226">SERVER-39850</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="700970">SERVER-39680</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="881088">SERVER-42540</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="935540">SERVER-43435</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="585021">SERVER-36498</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 11 Apr 2019 16:13:12 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 19 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-39680'>SERVER-39680</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1035</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>xgen-internal-githook</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 19 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jason.chan@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huq1xj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htyppj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2918">Repl 2019-04-22</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hupo6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>