<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:14:30 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-47549] changes stream can&apos;t keep casual consistency when move chunk happens</title>
                <link>https://jira.mongodb.org/browse/SERVER-47549</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I want to use &quot;changes stream&quot; to listen to all the change events from source sharding, and apply this data into target mongodb so that the target mongodb is just like a secondary of the primary. However, after I go through the source code, I think the changes stream can&apos;t keep casual consistency when move chunk happens. For a better explanation, I draw a picture as the attachment.&lt;/p&gt;

&lt;p&gt;In this picture, there are two shards, MongoS builds two cursors to fetch change stream event which is a transformation of oplog from MongoD. After that, MongoS runs the merge stage by &quot;ts+uuid+documentKey&quot; to sort the key. Everything is ok event move chunk happens because of the current hybrid logical time can keep the casual consistency. However, there is one corn case that may raise a bug: what if shard2 fetching speed is so fast that shard1, so that in my move chunk case, once shard2 has already fetch after &quot;ts=A2&quot;, but shard1 hasn&apos;t reach &quot;ts=A1&quot;. After shard1 catches up, the writing order of key A is &quot;set a = 2&quot; and then &quot;set a = 1&quot;, so the result of A is 2, but the real result is 1.&lt;/p&gt;

&lt;p&gt;There may be another case that lost data which isnot related to the previous example:&lt;br/&gt;
Since v3.6, MongoDB uses the logical time as timestamp which keeps the causal consistency. So different MongoS may have a different timestamp. In the change stream, MongoS uses local logical time if no afterClusterTime options given, will this cause some data loss? For example, mongos1 timestamp is 10:00, mongos2 is 10:02, shard1 is 09:59, shard2 is 10:01, if users send change stream command to mongos1, then mongos1 will send the aggregate command to each shard begin with 10:00, and the shard1 oplog from 09:59~10:00 will lost. I also find a jira &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31767&quot; title=&quot;Provide a window of snapshot history that is accessible for PIT reads&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31767&quot;&gt;&lt;del&gt;SERVER-31767&lt;/del&gt;&lt;/a&gt;(&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31767&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-31767&lt;/a&gt;). It means this problem has been solved since v4.1.1 by global point time, right?&lt;/p&gt;

&lt;p&gt;In my point of view, it&apos;ll be useful to add a &quot;wait policy&quot; on MongoS, for example, shard2 returns events with ts=05:10, after that MongoS cache it until all oplogs from other shards older than 05:10 to be received, and then, reply to user.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1315251">SERVER-47549</key>
            <summary>changes stream can&apos;t keep casual consistency when move chunk happens</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="backlog-server-triage">Backlog - Triage Team</assignee>
                                    <reporter username="cvinllen@gmail.com">vinllen chen</reporter>
                        <labels>
                    </labels>
                <created>Wed, 15 Apr 2020 03:56:01 +0000</created>
                <updated>Tue, 6 Dec 2022 02:29:46 +0000</updated>
                            <resolved>Wed, 15 Apr 2020 23:33:14 +0000</resolved>
                                                                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="3041720" author="carl.champain" created="Wed, 15 Apr 2020 23:31:54 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=cvinllen%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;cvinllen@gmail.com&quot;&gt;cvinllen@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;The right place for these questions is our &lt;a href=&quot;http://community.mongodb.com/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MongoDB Community Forums&lt;/a&gt; and I noticed that you have already created a post on the Forums &lt;a href=&quot;https://community.mongodb.com/t/the-dispatching-and-merging-policy-about-change-stream-in-mongos/2711&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. Someone should reply to your post soon.&lt;/p&gt;

&lt;p&gt;That said, the SERVER project is for bugs and feature suggestions for the MongoDB server. As this ticket does not appear to be a bug, I will now close it.&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
 Carl&lt;/p&gt;</comment>
                            <comment id="3040074" author="cvinllen@gmail.com" created="Wed, 15 Apr 2020 04:01:49 +0000"  >&lt;p&gt;In my point of view, it&apos;ll be useful to add a &quot;wait policy&quot; on MongoS, for example, shard2 returns events with ts=05:10, after that MongoS cache it until all oplogs from other shards older than 05:10 to be received, and then, reply to user.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="256032" name="example.png" size="256436" author="cvinllen@gmail.com" created="Wed, 15 Apr 2020 03:43:31 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25137"><![CDATA[Server Triage]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 15 Apr 2020 23:31:54 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 43 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 43 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-triage</customfieldvalue>
            <customfieldvalue>carl.champain@mongodb.com</customfieldvalue>
            <customfieldvalue>cvinllen@gmail.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxfbun:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hx2twn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxey3z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>