<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:26:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DRIVERS-2739] Add option to always use &quot;startAfter&quot; for changestreams</title>
                <link>https://jira.mongodb.org/browse/DRIVERS-2739</link>
                <project id="10980" key="DRIVERS">Drivers</project>
                    <description>&lt;div class=&quot;panel&quot; style=&quot;background-color: #fafbfc;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #fafbfc;&quot;&gt;
&lt;h3&gt;&lt;a name=&quot;Summary&quot;&gt;&lt;/a&gt;&lt;b&gt;Summary&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;Add option to resume change stream after receiving an &quot;invalidate&quot; event.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;Motivation&quot;&gt;&lt;/a&gt;&lt;b&gt;Motivation&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;After a change stream receives an &quot;invalidate&quot; event, the server closes the change stream.&lt;/p&gt;

&lt;p&gt;Callers wanting to continue receiving events are required to create a new change stream with the `startAfter` option set to the most recent `resumeToken`.&lt;/p&gt;

&lt;p&gt;An opt-in option (e.g. `ChangeStreamOptions.resumeAfterInvalidate`?) to resume using `startAfter` may provide a convenience.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Whoistheaffectedenduser%3F&quot;&gt;&lt;/a&gt;Who is the affected end user?&lt;/h4&gt;

&lt;p&gt;Change stream users who want to continue listening for events after &quot;invalidate&quot;.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Howdoesthisaffecttheenduser%3F&quot;&gt;&lt;/a&gt;How does this affect the end user?&lt;/h4&gt;

&lt;p&gt;To achieve this behavior, it requires a workaround: Track the resume token. If an &quot;invalidate&quot; event is received, recreate the change stream with the `startAfter` option.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Howlikelyisitthatthisproblemorusecasewilloccur%3F&quot;&gt;&lt;/a&gt;How likely is it that this problem or use case will occur?&lt;/h4&gt;

&lt;p&gt;Not sure.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Iftheproblemdoesoccur%2Cwhataretheconsequencesandhowseverearethey%3F&quot;&gt;&lt;/a&gt;If the problem does occur, what are the consequences and how severe are they?&lt;/h4&gt;

&lt;p&gt;Requires a workaround.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisissueurgent%3F&quot;&gt;&lt;/a&gt;Is this issue urgent?&lt;/h4&gt;

&lt;p&gt;No. There is a workaround.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisticketrequiredbyadownstreamteam%3F&quot;&gt;&lt;/a&gt;Is this ticket required by a downstream team?&lt;/h4&gt;

&lt;p&gt;Requested by internal team described in this comment.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisticketonlyfortests%3F&quot;&gt;&lt;/a&gt;Is this ticket only for tests?&lt;/h4&gt;

&lt;p&gt;No.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;AcceptanceCriteria&quot;&gt;&lt;/a&gt;&lt;b&gt;Acceptance Criteria&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;Enable change streams to continue listening after receiving an &quot;invalidate&quot; event.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;
&lt;hr /&gt;
&lt;p&gt;Original ticket description:&lt;br/&gt;
&lt;blockquote&gt;&lt;p&gt;Modifying the example, this behavior reproduces:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Insert a document to&#160;&lt;tt&gt;db.coll&lt;/tt&gt;&#160;to create the collection.&lt;/li&gt;
	&lt;li&gt;Watch on&#160;&lt;tt&gt;db.coll&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;Drop&#160;&lt;tt&gt;db.coll&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;Iterate. Get a &#8220;drop&#8221; event.&lt;/li&gt;
	&lt;li&gt;Iterate. Get an &#8220;invalidate&#8221; event.&lt;/li&gt;
	&lt;li&gt;Iterate. Get an error: &#8220;Attempting to resume a change stream using &#8216;resumeAfter&#8217; is not allowed from an invalidate notification&#8221;.The&#160;&lt;a href=&quot;https://github.com/mongodb/specifications/blob/master/source/change-streams/change-streams.rst#resume-process&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;change stream specification&lt;/a&gt;&#160;describes the expected protocol. Drivers do not inspect events or handle &#8220;invalidate&#8221; specially.To have a change stream survive after an &#8220;invalidate&#8221;, I expect it is needed to recreate the change stream with the &#8220;startAfter&#8221; option.&lt;br/&gt;
&#160;&lt;br/&gt;
Kevin Albertson&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696003304780949?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;2 hours ago&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I think there is a related bug in the C driver. I do not expect the resume after &#8220;invalidate&#8221;. Comparing with the Go driver, I expect iterating after &#8220;invalidate&#8221; should result in no event returned and no error. I plan to file a ticket.&lt;br/&gt;
&#160;&lt;br/&gt;
matthew.normyle&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696007583151659?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;37 minutes ago&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Thank you so much for investigating.&lt;/p&gt;&lt;/blockquote&gt;To have a change stream survive after an &#8220;invalidate&#8221;, I expect it is needed to recreate the change stream with the &#8220;startAfter&#8221; option.&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Yep, for what it&#8217;s worth, this problem goes away if I change this&#160;&#160;&lt;a href=&quot;https://github.com/mongodb/mongo-c-driver/blob/master/src/libmongoc/src/mongoc/mongoc-change-stream.c#L152&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;line&lt;/a&gt;&#160;to&#160;&lt;tt&gt;startAfter&lt;/tt&gt;&#160;instead of&#160;&lt;tt&gt;resumeAfter&lt;/tt&gt;&#160; &lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/1f642@2x.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;br/&gt;
&#160;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-c-driver/blob/master/src/libmongoc/src/mongoc/mongoc-change-stream.c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongoc-change-stream.c&lt;/a&gt;&lt;br/&gt;
&#160; &#160; &#160; &#160; &#160; &#160; &#160; &amp;amp;change_stream_doc, &quot;resumeAfter&quot;, &amp;amp;stream-&amp;gt;resume_token);&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://slack-imgs.com/?c=1&amp;amp;o1=wi32.he32.si&amp;amp;url=https%3A%2F%2Fslack.github.com%2Fstatic%2Fimg%2Ffavicon-neutral.png&quot; height=&quot;16&quot; width=&quot;16&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;a href=&quot;https://github.com/mongodb/mongo-c-driver&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb/mongo-c-driver&lt;/a&gt;&#160;|&#160;Added by&#160;&lt;a href=&quot;https://mongodb.slack.com/services/B01UWPSL9L3&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;GitHub&lt;/a&gt;&lt;br/&gt;
&#160;&lt;br/&gt;
Kevin Albertson&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696008315151819?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;25 minutes ago&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I expect the Resume Process using&#160;&lt;tt&gt;resumeAfter&lt;/tt&gt;&#160;rather than&#160;&lt;tt&gt;startAfter&lt;/tt&gt;&#160;is intentional. After &#8220;invalidate&#8221;, later events may not be related to previous events. E.g. the collection may have been dropped, and recreated with a different schema. It may no longer be the &#8220;same collection&#8221;.&lt;br/&gt;
&#160;&lt;br/&gt;
matthew.normyle&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696008373260199?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;24 minutes ago&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Yep, totally makes sense. Perhaps the fix here is to expose an option for this, so the customer can decide what happens. In our case we&#8217;d like to keep reading after a drop and invalidate event (&lt;tt&gt;startAfter&lt;/tt&gt;&#160;semantics).&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://ca.slack-edge.com/E01C4Q4H3CL-U04EC6V8F8U-925ed45be79c-48&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;br/&gt;
matthew.normyle&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696008409295079?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;24 minutes ago&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;(Also I think startAfter only works for MongoDB &amp;gt;= 4.X)&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://ca.slack-edge.com/E01C4Q4H3CL-U1D17KD24-742a3597eb1b-48&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;br/&gt;
Kevin Albertson&#160;&#160;&lt;a href=&quot;https://mongodb.slack.com/archives/C0V2T72R2/p1696008503184649?thread_ts=1695942529.179219&amp;amp;cid=C0V2T72R2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;22 minutes ago&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That seems reasonable. If it is wanted, I suggest filing a DRIVERS ticket with the feature request. That may result in a specification change to support that use case.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;&lt;/p&gt;
&lt;/div&gt;&lt;/div&gt;</description>
                <environment></environment>
        <key id="2454917">DRIVERS-2739</key>
            <summary>Add option to always use &quot;startAfter&quot; for changestreams</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="10049" iconUrl="https://jira.mongodb.org/images/icons/statuses/information.png" description="">Needs Triage</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="matthew.normyle@mongodb.com">Matthew Normyle</reporter>
                        <labels>
                    </labels>
                <created>Fri, 29 Sep 2023 17:51:34 +0000</created>
                <updated>Mon, 2 Oct 2023 19:24:29 +0000</updated>
                                                                <component>Change Streams</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                    <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2456046">CDRIVER-4738</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                            <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10250"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10951" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Driver Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10748"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2anik:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>