<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:20:33 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-49691] Change streams may be subject to spurious &quot;CappedPositionLost&quot; when resuming</title>
                <link>https://jira.mongodb.org/browse/SERVER-49691</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Our testing infrastructure uncovered a rare case where this might happen, detailed in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-49690&quot; title=&quot;Retry &amp;quot;CappedPositionLost&amp;quot; when searching for old oplog entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-49690&quot;&gt;&lt;del&gt;SERVER-49690&lt;/del&gt;&lt;/a&gt;. When a change stream is resuming, it may encounter this error. As far as I know this has never been observed, but I see no reason it couldn&apos;t happen. I would recommend looking into whether we can reproduce this. If so, I think we should do one of the following:&lt;br/&gt;
1) Disabling yielding when doing the oplog check upon resume during the change stream&lt;br/&gt;
2) Adding a similar retry loop within the change stream&lt;br/&gt;
3) Ensuring drivers will retry this error&lt;/p&gt;

&lt;p&gt;During &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-49690&quot; title=&quot;Retry &amp;quot;CappedPositionLost&amp;quot; when searching for old oplog entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-49690&quot;&gt;&lt;del&gt;SERVER-49690&lt;/del&gt;&lt;/a&gt; I looked into option #1 but the patch quickly exploded. I&apos;ll attach my WIP but it certainly won&apos;t compile and doesn&apos;t plumb the yield policy far enough to fix the issue.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1413466">SERVER-49691</key>
            <summary>Change streams may be subject to spurious &quot;CappedPositionLost&quot; when resuming</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="bernard.gorman@mongodb.com">Bernard Gorman</assignee>
                                    <reporter username="charlie.swanson@mongodb.com">Charlie Swanson</reporter>
                        <labels>
                    </labels>
                <created>Fri, 17 Jul 2020 16:50:24 +0000</created>
                <updated>Fri, 27 Oct 2023 20:42:10 +0000</updated>
                            <resolved>Thu, 27 Aug 2020 13:34:34 +0000</resolved>
                                                                    <component>Aggregation Framework</component>
                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3362936" author="charlie.swanson" created="Thu, 27 Aug 2020 19:17:38 +0000"  >&lt;p&gt;Sorry this dropped off my radar! I suspect this may still be a problem. If I remember correctly, the problem wasn&apos;t so much that we were doing a separate sub-pipeline. It was just that we could establish a cursor, yield, and then the collection would be truncated. All of this before we examine the first record. I don&apos;t see anything about &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-48523&quot; title=&quot;Unconditionally check the first entry in the oplog when attempting to resume a change stream&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-48523&quot;&gt;&lt;del&gt;SERVER-48523&lt;/del&gt;&lt;/a&gt; that changes this dynamic? As long as yielding is around, I think this may always be a problem since the collection may be truncated during a yield? Thinking more, maybe the most practical solution is to have our drivers retry this error if it is encountered during a resume (since I don&apos;t think it&apos;s fatal). It might be nice for us to do this internally so our tests don&apos;t have to worry about it though.&lt;/p&gt;</comment>
                            <comment id="3361968" author="bernard.gorman" created="Thu, 27 Aug 2020 13:34:07 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt;: now that we&apos;ve pushed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-48523&quot; title=&quot;Unconditionally check the first entry in the oplog when attempting to resume a change stream&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-48523&quot;&gt;&lt;del&gt;SERVER-48523&lt;/del&gt;&lt;/a&gt;, which removes the separate oplog check that you highlighted above, I&apos;m going to close this as &quot;Gone Away.&quot;&lt;/p&gt;</comment>
                            <comment id="3290561" author="bernard.gorman" created="Fri, 17 Jul 2020 19:24:15 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt;: ah, I see - thanks for the clarification! I think the scenario you&apos;re concerned about will be indirectly addressed by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-48523&quot; title=&quot;Unconditionally check the first entry in the oplog when attempting to resume a change stream&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-48523&quot;&gt;&lt;del&gt;SERVER-48523&lt;/del&gt;&lt;/a&gt;, by making the oplog check part of the actual &lt;tt&gt;COLLSCAN&lt;/tt&gt; rather than a separate operation. Does that make sense?&lt;/p&gt;</comment>
                            <comment id="3290433" author="charlie.swanson" created="Fri, 17 Jul 2020 18:14:02 +0000"  >&lt;p&gt;Sorry I think I should be more specific &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bernard.gorman&quot; class=&quot;user-hover&quot; rel=&quot;bernard.gorman&quot;&gt;bernard.gorman&lt;/a&gt;. I was thinking of &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3bfbd4f8add70ec14a71f6907bafe20f648fc4bc/src/mongo/db/pipeline/document_source_check_resume_token.cpp#L328&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this code&lt;/a&gt;. Here we are simply trying to determine if there&apos;s enough oplog history, but we might get this spurious error. If I&apos;m not mistaken, that call to &lt;tt&gt;getNext()&lt;/tt&gt; may just throw the &lt;tt&gt;CappedPositionLost&lt;/tt&gt; error. &lt;/p&gt;</comment>
                            <comment id="3290348" author="bernard.gorman" created="Fri, 17 Jul 2020 17:36:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt;: I don&apos;t think attempting to resume after this error is ever appropriate. When resuming, a change stream will use the &lt;tt&gt;minTs&lt;/tt&gt; mechanism to skip directly to the timestamp of the resume token in the oplog (or the point immediately before it, if &lt;tt&gt;startAtOperationTime&lt;/tt&gt; is used and no event with the specified timestamp exists). If we yield and the oplog rolls over this point to produce a &lt;tt&gt;CappedPositionLost&lt;/tt&gt; exception, it implies that the stream has genuinely become unresumable.&lt;/p&gt;

&lt;p&gt;I think option (1) is the only improvement we could make here, to avoid the situation where we successfully establish the cursor but then yield and roll over instead of scanning away from the start of the oplog. But we would need to account for the possibility that the oplog contains no events (or very infrequent events) after the resume point, since this could result in a very lengthy scan that never yields.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1413459">SERVER-49690</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1368228">SERVER-48523</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="270364" name="wip_plumb_yield_policy.patch" size="1405" author="charlie.swanson@mongodb.com" created="Fri, 17 Jul 2020 16:50:03 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 17 Jul 2020 17:36:12 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 23 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 23 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>
            <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxvih3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hxif0f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3612">Query 2020-08-24</customfieldvalue>
    <customfieldvalue id="3614">Query 2020-09-07</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxv4qf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>