<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:37:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-34702] Whole-DB or whole-cluster change streams may not provide a total ordering if resumed after a drop AND resume token does not include the shard key</title>
                <link>https://jira.mongodb.org/browse/SERVER-34702</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This can happen in two separate scenarios.&lt;/p&gt;

&lt;h3&gt;&lt;a name=&quot;Scenario1&quot;&gt;&lt;/a&gt;Scenario 1&lt;/h3&gt;
&lt;ol&gt;
	&lt;li&gt;A whole-db or whole-cluster change stream is opened.&lt;/li&gt;
	&lt;li&gt;An insert notification is returned, which does not yet include a shard key because the collection has not been sharded yet.&lt;/li&gt;
	&lt;li&gt;The collection from the insert is sharded.&lt;/li&gt;
	&lt;li&gt;A chunk migrates to another shard.&lt;/li&gt;
	&lt;li&gt;Two inserts happen with the same cluster time &lt;em&gt;and&lt;/em&gt; the same _id, one on each shard.&lt;/li&gt;
	&lt;li&gt;The collection is dropped.&lt;/li&gt;
	&lt;li&gt;The change stream is resumed, using the resume token from the first insert, at step 2.&lt;/li&gt;
&lt;/ol&gt;


&lt;h3&gt;&lt;a name=&quot;Scenario2&quot;&gt;&lt;/a&gt;Scenario 2&lt;/h3&gt;
&lt;ol&gt;
	&lt;li&gt;A whole-db or whole-cluster change stream is opened, but at least one shard which has a chunk for collection with UUID &apos;x&apos; does not yet return any results for &apos;x&apos;.&lt;/li&gt;
	&lt;li&gt;No updates or deletes happen on that shard, but an insert on &apos;x&apos; happens on that shard and is not yet returned from the stream.&lt;/li&gt;
	&lt;li&gt;Collection with UUID &apos;x&apos; is dropped, causing us to lose information about the shard key.&lt;/li&gt;
	&lt;li&gt;The insert notification is returned, and will not contain the shard key.&lt;/li&gt;
	&lt;li&gt;Two inserts happen with the same cluster time &lt;em&gt;and&lt;/em&gt; the same _id (this can only happen on two different shards).&lt;/li&gt;
	&lt;li&gt;The change stream is resumed, using the resume token from the first insert, at step 2.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;In any scenario when the stream is resumed no shard will know the shard key because the collection has been dropped, and the resume token will not include it. Because of this, each shard will produce notifications for the two matching inserts with identical resume tokens. Both will have the same timestamp, same UUID, and same _id.&lt;/p&gt;

&lt;p&gt;The impact of this lack of ordering is that if the client then tried to resume using one of those tokens, they would potentially get the same notification twice or skip one of the inserts.&lt;/p&gt;

&lt;p&gt;Once opened, a change stream will learn the shard key from any oplog entry which contains it. So if there were an update or a delete on the same collection, the change stream would learn the shard key for that collection and avoid this bug. Insert notifications do not include enough information to figure out the shard key. This means that in order to possibly be affected by this issue, the following must be true:&lt;/p&gt;

&lt;h4&gt;&lt;a name=&quot;Tobeimpactedbyscenario1&quot;&gt;&lt;/a&gt;To be impacted by scenario 1&lt;/h4&gt;
&lt;ul&gt;
	&lt;li&gt;You must be resuming with a resume token generated before the change stream detected the collection has been sharded. This must be before the first chunk migration.&lt;/li&gt;
	&lt;li&gt;There must have been at least one chunk migration.&lt;/li&gt;
	&lt;li&gt;There must have been at least two inserts with the same cluster time, and the same &lt;tt&gt;_id&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;There must &lt;b&gt;not&lt;/b&gt; have been any updates or deletes that happened after the chunk migration but before the insert.&lt;/li&gt;
	&lt;li&gt;The client must try to resume using the resume token from one of the inserts with a duplicate resume token.&lt;/li&gt;
	&lt;li&gt;The collection has been dropped at the time of both resumes.&lt;/li&gt;
&lt;/ul&gt;


&lt;h4&gt;&lt;a name=&quot;Tobeimpactedbyscenario2&quot;&gt;&lt;/a&gt;To be impacted by scenario 2&lt;/h4&gt;
&lt;ul&gt;
	&lt;li&gt;An insert notification must be the first change seen in the stream for a collection &apos;x&apos; on at least one shard.&lt;/li&gt;
	&lt;li&gt;The collection must have been dropped when the change stream observes that insert.&lt;/li&gt;
	&lt;li&gt;There must be a change on another shard with the same cluster time as that insert.&lt;/li&gt;
	&lt;li&gt;The client must try to resume using the resume token from one of the changes with a duplicate cluster time.&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="534717">SERVER-34702</key>
            <summary>Whole-DB or whole-cluster change streams may not provide a total ordering if resumed after a drop AND resume token does not include the shard key</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="charlie.swanson@mongodb.com">Charlie Swanson</reporter>
                        <labels>
                            <label>change-streams-improvements</label>
                            <label>query-44-grooming</label>
                    </labels>
                <created>Thu, 26 Apr 2018 18:42:53 +0000</created>
                <updated>Fri, 27 Oct 2023 20:43:28 +0000</updated>
                            <resolved>Mon, 6 Mar 2023 10:19:33 +0000</resolved>
                                                                    <component>Aggregation Framework</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="5251211" author="bernard.gorman" created="Mon, 6 Mar 2023 10:19:21 +0000"  >&lt;p&gt;This ticket was written at a time when the &lt;tt&gt;insert&lt;/tt&gt; oplog entry did not contain the &lt;tt&gt;documentKey&lt;/tt&gt;, and we were therefore reliant on either reading the collection sharding state, which is lost when the collection is dropped, or on seeing an oplog entry that &lt;b&gt;&lt;em&gt;does&lt;/em&gt;&lt;/b&gt; contain the shard key, such as an update or delete. After &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-61892&quot; title=&quot;Replace documentKey paths cache with document key from the oplog&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-61892&quot;&gt;&lt;del&gt;SERVER-61892&lt;/del&gt;&lt;/a&gt;, however, &lt;tt&gt;insert&lt;/tt&gt; oplog entries themselves contain the document key, and the above scenarios can therefore no longer occur. Closing as Gone Away.&lt;/p&gt;</comment>
                            <comment id="1875375" author="charlie.swanson" created="Thu, 26 Apr 2018 18:48:30 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34088&quot; title=&quot;Add ability to watch all changes for an entire sharded cluster&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34088&quot;&gt;&lt;del&gt;SERVER-34088&lt;/del&gt;&lt;/a&gt; skipped a check that would assert the UUID exists upon resuming for whole-db or whole-cluster change streams, which is why this only affects those change streams, and not single-collection streams.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="534734">SERVER-34705</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="515629">SERVER-34088</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sun, 17 May 2020 15:01:34 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        48 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1951</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            48 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>
            <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htwjgv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr27rr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htw5p3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>