<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:44:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36771] [post-project] Provide a way to know when there are no active transactions in a sharded cluster</title>
                <link>https://jira.mongodb.org/browse/SERVER-36771</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This is required for Atlas Live Import to be able to maintain transactional integrity.&lt;/p&gt;</description>
                <environment></environment>
        <key id="591239">SERVER-36771</key>
            <summary>[post-project] Provide a way to know when there are no active transactions in a sharded cluster</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</assignee>
                                    <reporter username="judah.schvimer@mongodb.com">Judah Schvimer</reporter>
                        <labels>
                    </labels>
                <created>Mon, 20 Aug 2018 21:03:47 +0000</created>
                <updated>Tue, 4 Dec 2018 12:35:40 +0000</updated>
                            <resolved>Tue, 4 Dec 2018 12:35:31 +0000</resolved>
                                                                    <component>Replication</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="2079660" author="kaloian.manassiev" created="Tue, 4 Dec 2018 12:35:10 +0000"  >&lt;p&gt;Based on a discussion with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=shane.harvey&quot; class=&quot;user-hover&quot; rel=&quot;shane.harvey&quot;&gt;shane.harvey&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, the &lt;a href=&quot;https://docs.mongodb.com/manual/reference/command/serverStatus/#serverstatus.transactions.currentOpen&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;transactions.currentOpen&lt;/tt&gt;&lt;/a&gt; metric will be sufficient for figuring out when the system is quiesced of active transactions, therefore there is nothing for the Sharding Team to do.&lt;/p&gt;</comment>
                            <comment id="2061802" author="judah.schvimer" created="Wed, 14 Nov 2018 16:00:10 +0000"  >&lt;blockquote&gt;
&lt;p&gt;From the description in the linked mongomirror ticket, it looks like mongomirror will buffer prepared transactions. Is this the right thing to do rather than just have mongomirror pass all oplog entries to the destination cluster&apos;s primary and let the replication machinery deal with prepare/abort and commit?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;Making &lt;tt&gt;applyOps&lt;/tt&gt; work well with prepare would be incredibly difficult due to initial-sync-like idempotency problems that &lt;tt&gt;mongomirror&apos;s&lt;/tt&gt; push based initial sync hits. We would have to reconstruct oplog pointers, change &quot;commit timestamps&quot;, and log a new type of &lt;tt&gt;applyOps&lt;/tt&gt; so that secondaries know to relax constraints in the same way the primary did. It was deemed far easier to implement this in &lt;tt&gt;mongomirror&lt;/tt&gt; itself.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;That way, after mongomirror has applyOp&apos;d all oplog entries to the shards&apos; primaries, it can just log a no-op oplog entry and wait for that to majority propagate. From that point onward the two clusters are identical.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;How does this require actually sending prepare rather than buffering transactions? Cross-shard transactions write their own oplog entries even when the user has stopped writing, and it takes time for those oplog entries to reach all secondaries, so simply applying everything that&apos;s there as of when the user stops writing is not sufficient. Am I misunderstanding?&lt;/p&gt;</comment>
                            <comment id="2061201" author="kaloian.manassiev" created="Wed, 14 Nov 2018 08:31:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=shane.harvey&quot; class=&quot;user-hover&quot; rel=&quot;shane.harvey&quot;&gt;shane.harvey&lt;/a&gt;, I would like to understand a little bit better about the requirements here.&lt;/p&gt;

&lt;p&gt;Presumably, as part of the import process, the source cluster will at some point be quiesced so that mongomirror can catch-up and this ticket is about figuring out when all active transactions have finished being applied there - is this correct?&lt;/p&gt;

&lt;p&gt;Like you say &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, waiting for zero transactions will not be sufficient, because there can be &quot;lulls&quot; in the oplog where there are no transactions which can mistakenly be interpreted.&lt;/p&gt;

&lt;p&gt;From the description in the linked mongomirror ticket, it looks like mongomirror will buffer prepared transactions. Is this the right thing to do rather than just have mongomirror pass all oplog entries to the destination cluster&apos;s primary and let the replication machinery deal with prepare/abort and commit?&lt;/p&gt;

&lt;p&gt;That way, after mongomirror has &lt;tt&gt;applyOp&lt;/tt&gt;&apos;d all oplog entries to the shards&apos; primaries, it can just log a no-op oplog entry and wait for that to majority propagate. From that point onward the two clusters are identical.&lt;/p&gt;

&lt;p&gt;By the way, at that point, how can there be prepared transactions? This would mean that the source cluster was quiesced at the wrong point in time.&lt;/p&gt;</comment>
                            <comment id="1980681" author="judah.schvimer" created="Mon, 20 Aug 2018 21:20:23 +0000"  >&lt;p&gt;It may be sufficient to simply wait for every node in the cluster to have 0 active transactions, but if one node is super lagged that may not be sufficient. Maybe we have to wait for the replication lag to actually go to 0 on all nodes or surface a &quot;lastActiveTransactionNumber&quot; field and wait for that to be the same on each node as well. We have to be careful that the nodes can&apos;t just roll back the last transaction commit right after the metric threshold is met.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10020">
                    <name>Gantt Dependency</name>
                                                                <inwardlinks description="has to be done after">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 14 Nov 2018 08:31:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 10 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-834</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 10 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu5zmn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr84qn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2581">Sharding 2018-11-19</customfieldvalue>
    <customfieldvalue id="2639">Sharding 2018-12-17</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu5lvz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>