<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:29:58 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32349] Resuming a sharded change stream when there are multiple changes with the same timestamp may be impossible</title>
                <link>https://jira.mongodb.org/browse/SERVER-32349</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When resuming a change stream, we first need to make sure that the oplog has enough history to allow a resume. To do this, we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.6.0-rc4/src/mongo/db/pipeline/document_source_check_resume_token.cpp#L56-L74&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;make sure that the first thing in the stream is the resume token we expect&lt;/a&gt;. This works correctly in an unsharded environment, because we will start the oplog query with a &lt;tt&gt;ts: {$gte: &amp;lt;resume ts&amp;gt;&lt;/tt&gt;} query, and each change&apos;s timestamp will be unique, so the first thing that comes back should be the change we&apos;re looking to resume after (if it&apos;s still possible to resume).&lt;/p&gt;

&lt;p&gt;Things are a bit different in a sharded scenario. To start, we don&apos;t know which shard is going to have the resume token, so we need to perform the check against the first document &lt;em&gt;after&lt;/em&gt; merging the results from each shard. Secondly, the timestamp of each change is not guaranteed to be unique in a sharded cluster, two changes can happen &apos;simultaneously&apos; (with the same timestamp) on two different shards. In situations like this, it&apos;s possible and legal that the first thing in the change stream will not be the resume token we&apos;re looking for, but rather a change that preceded the change we&apos;re resuming after and happened to have the same timestamp.&lt;/p&gt;

&lt;p&gt;To account for this, the logic that checks if we are able to resume (inside &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.6.0-rc4/src/mongo/db/pipeline/document_source_check_resume_token.cpp#L56&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;DocumentSourceEnsureResumeTokenPresent::getNext()&lt;/tt&gt;&lt;/a&gt;) should allow arbitrarily many changes to occur before the resume token iff they have the same timestamp as the resume token. As soon as we see the resume token itself we know it&apos;s possible to resume, or if we see a change with a higher timestamp we know it&apos;s not possible to resume.&lt;/p&gt;</description>
                <environment></environment>
        <key id="472498">SERVER-32349</key>
            <summary>Resuming a sharded change stream when there are multiple changes with the same timestamp may be impossible</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="martin.neupauer@mongodb.com">Martin Neupauer</assignee>
                                    <reporter username="charlie.swanson@mongodb.com">Charlie Swanson</reporter>
                        <labels>
                    </labels>
                <created>Thu, 14 Dec 2017 19:54:00 +0000</created>
                <updated>Mon, 30 Oct 2023 23:09:53 +0000</updated>
                            <resolved>Mon, 22 Jan 2018 14:11:04 +0000</resolved>
                                    <version>3.6.0</version>
                                    <fixVersion>3.6.3</fixVersion>
                    <fixVersion>3.7.2</fixVersion>
                                    <component>Aggregation Framework</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="1803705" author="xgen-internal-githook" created="Mon, 12 Feb 2018 23:40:42 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;martin.neupauer@10gen.com&apos;, &apos;name&apos;: &apos;Martin Neupauer&apos;, &apos;username&apos;: &apos;MartinNeupauer&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32349&quot; title=&quot;Resuming a sharded change stream when there are multiple changes with the same timestamp may be impossible&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-32349&quot;&gt;&lt;del&gt;SERVER-32349&lt;/del&gt;&lt;/a&gt; Change streams over sharded collections may produce merged op log entries&lt;br/&gt;
with the same timestamps if the operations are coming from multiple shards. When we&lt;br/&gt;
resume the change stream we have to position to the right place - the position is determined&lt;br/&gt;
both by the timestamp and the document id. Previously we checked the timestamp only,&lt;br/&gt;
now we loop over the equal timestamps and find the right document.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 194ec4857fa0db8085da88e22eaae96687902d66)&lt;br/&gt;
Branch: v3.6&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e74bf273ef83cfbee332fe33079c0f640c71f7bb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e74bf273ef83cfbee332fe33079c0f640c71f7bb&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1779696" author="xgen-internal-githook" created="Fri, 19 Jan 2018 22:44:11 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Martin Neupauer&apos;, &apos;email&apos;: &apos;martin.neupauer@10gen.com&apos;, &apos;username&apos;: &apos;MartinNeupauer&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32349&quot; title=&quot;Resuming a sharded change stream when there are multiple changes with the same timestamp may be impossible&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-32349&quot;&gt;&lt;del&gt;SERVER-32349&lt;/del&gt;&lt;/a&gt; Change streams over sharded collections may produce merged op log entries&lt;br/&gt;
with the same timestamps if the operations are coming from multiple shards. When we&lt;br/&gt;
resume the change stream we have to position to the right place - the position is determined&lt;br/&gt;
both by the timestamp and the document id. Previously we checked the timestamp only,&lt;br/&gt;
now we loop over the equal timestamps and find the right document.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/194ec4857fa0db8085da88e22eaae96687902d66&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/194ec4857fa0db8085da88e22eaae96687902d66&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1766639" author="charlie.swanson" created="Fri, 5 Jan 2018 21:14:02 +0000"  >&lt;p&gt;As far as I know, the fix described in the description should work - most of the work would probably be in creating a test for this, though even that shouldn&apos;t be crazy difficult. I would estimate a couple days, maybe 2?&lt;/p&gt;</comment>
                            <comment id="1766374" author="david.storch" created="Fri, 5 Jan 2018 18:05:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt; can you provide a back of the envelope estimate on how many days of developer time you think this will require?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 5 Jan 2018 18:05:23 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>martin.neupauer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htma4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr9c3r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2077">Query 2018-01-15</customfieldvalue>
    <customfieldvalue id="2096">Query 2018-01-29</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htlw93:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>