<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:06:39 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-66870] Improvements to resume token format</title>
                <link>https://jira.mongodb.org/browse/SERVER-66870</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;There are a number of small improvements we could make to the resume token format, although these may require a bump in the version number. For instance:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Never explicitly encode any of the redundant info beyond &lt;tt&gt;tokenType&lt;/tt&gt; for HWM tokens.&lt;/li&gt;
	&lt;li&gt;Encode &lt;tt&gt;tnxOpIndex&lt;/tt&gt; as explicitly null for non-txn events. This will generally ensure that txn events from the same shard are contiguous in the stream; currently, the first entry in a txn will interleave with any events from other shards at the same clusterTime.&lt;/li&gt;
	&lt;li&gt;Consider hashing the &lt;tt&gt;eventIdentifier&lt;/tt&gt; as SHA256 or similar. This would limit the resume token to a fixed size, but we would lose some potentially valuable diagnostic information in the process.&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2058417">SERVER-66870</key>
            <summary>Improvements to resume token format</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="bernard.gorman@mongodb.com">Bernard Gorman</reporter>
                        <labels>
                    </labels>
                <created>Tue, 31 May 2022 13:11:07 +0000</created>
                <updated>Mon, 25 Sep 2023 11:13:16 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="5554080" author="bernard.gorman" created="Mon, 10 Jul 2023 12:01:57 +0000"  >&lt;p&gt;While we&apos;re considering these changes, we should also examine how HWMs are generated on &lt;tt&gt;mongoD&lt;/tt&gt;. Currently, if we hit EOF &lt;b&gt;&lt;em&gt;and&lt;/em&gt;&lt;/b&gt; the last-seen oplog timestamp has advanced since our last read, we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e4412ccb7678b797757eefb9e34839a76ab5ccfb/src/mongo/db/pipeline/plan_executor_pipeline.cpp#L202-L209&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;set the HWM to that oplog timestamp and generate a PBRT using it&lt;/a&gt;. We do this because in this scenario, we know that the last entry in the oplog does not match our filter. But because HWM tokens sort before events at the same &lt;tt&gt;clusterTime&lt;/tt&gt;, this means that we have effectively set the PBRT to just &lt;b&gt;&lt;em&gt;before&lt;/em&gt;&lt;/b&gt; the last event in the oplog, which is somewhat counter-intuitive. It also means that if the HWM were used to start a &lt;em&gt;different&lt;/em&gt; pipeline, it could end up returning this event. This would not be incorrect behaviour; by definition, the event was not observed on the original stream, so we would not be returning a repeated event. But the semantics are subtle and slightly awkward to remember.&lt;/p&gt;

&lt;p&gt;An alternative would be to set the HWM to one tick beyond the most recent event in the oplog. It seems unintuitive to set the HWM to a timestamp that doesn&apos;t yet exist in the oplog, but it&apos;s &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e4412ccb7678b797757eefb9e34839a76ab5ccfb/src/mongo/db/pipeline/document_source_change_stream.cpp#L255-L257&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;similar to what we do when opening a new stream&lt;/a&gt;. We would have to consider whether this change in behaviour would have ramifications elsewhere, e.g. &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e4412ccb7678b797757eefb9e34839a76ab5ccfb/src/mongo/db/pipeline/change_stream_event_transform.cpp#L162-L165&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;when resuming a stream with a HWM of a different token version than the default&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        30 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1951</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            30 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0wusn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0fqm0:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0wgxz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>