<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:33:17 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-54350] Investigate potential race conditions in SBE oplog plans</title>
                <link>https://jira.mongodb.org/browse/SERVER-54350</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In SBE, we generate special optimized plans in the case where we are running a collection scan on the oplog. A number of these plans involve either resolving a &lt;tt&gt;recordId&lt;/tt&gt; in advance, or performing two separate scans within the same execution plan; one scan which checks some condition or produces some output, and a second &quot;real&quot; scan which uses that output as a parameter of its own execution. During development of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50580&quot; title=&quot;SBE should obey ASSERT_MIN_TS_HAS_NOT_FALLEN_OFF_OPLOG flag&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50580&quot;&gt;&lt;del&gt;SERVER-50580&lt;/del&gt;&lt;/a&gt;, we realised that it may be possible for these optimized oplog plans to behave incorrectly if entries fall off the oplog in the latency between the time we run the first part of the plan and the time we begin executing the &quot;real&quot; scan.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/b3c26f8c1403e499f4a996d684b298d3f3b1f12f/src/mongo/db/query/sbe_stage_builder_coll_scan.cpp#L142-L159&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Here&lt;/a&gt;, for instance, we resolve the &lt;tt&gt;recordId&lt;/tt&gt; of the entry to which we want to skip before constructing the SBE plan, and then &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b3c26f8c1403e499f4a996d684b298d3f3b1f12f/src/mongo/db/query/sbe_stage_builder_coll_scan.cpp#L183-L201&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;inject it into the plan&lt;/a&gt; as a constant value. But it may be possible that, in the time between the point at which we resolve the &lt;tt&gt;seekRecordId&lt;/tt&gt; and the point at which we actually begin executing the scan, that record has fallen off the oplog. If this happens, we will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b3c26f8c1403e499f4a996d684b298d3f3b1f12f/src/mongo/db/exec/sbe/stages/scan.cpp#L222-L228&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;incorrectly EOF the scan immediately&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Similarly, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b3c26f8c1403e499f4a996d684b298d3f3b1f12f/src/mongo/db/query/sbe_stage_builder_coll_scan.cpp#L261-L289&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; we create a NLJ whose outer branch scans the oplog until it reaches the first entry that matches our filter, and then passes that &lt;tt&gt;recordId&lt;/tt&gt; to the inner branch, which continues the scan from that point without applying the filter to any subsequent entries. But if that &lt;tt&gt;recordId&lt;/tt&gt; falls off the oplog between the time the first scan completes and the time the second scan begins (including, but potentially not limited to, the case where we yield at the wrong moment), we will again hit a spurious EOF. The same is true of the ASSERT_MIN_TS &lt;tt&gt;UnionStage&lt;/tt&gt; plan proposed in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50580&quot; title=&quot;SBE should obey ASSERT_MIN_TS_HAS_NOT_FALLEN_OFF_OPLOG flag&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50580&quot;&gt;&lt;del&gt;SERVER-50580&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;We do not believe that tailable cursors in general are susceptible to this problem, despite using a two-scan plan, because the first scan always scans to EOF before passing the last observed &lt;tt&gt;recordId&lt;/tt&gt; to the second scan. The user will have to issue a &lt;tt&gt;getMore&lt;/tt&gt; before the second branch is executed; if the &lt;tt&gt;recordId&lt;/tt&gt; has fallen off the capped collection by then, the plan will throw &lt;tt&gt;CappedPositionLost&lt;/tt&gt;. Tailable &lt;tt&gt;awaitData&lt;/tt&gt; cursors may be more susceptible, since they will continue to attempt to pull from the second branch after the first branch EOFs.&lt;/p&gt;

&lt;p&gt;The way to resolve the &quot;double scan&quot; scenario would be to incorporate the filtering performed by the first scan directly into the second, so that it is executed inline with the &quot;real&quot; scan; this means that there would be no inter-scan latency window during which entries could unexpectedly fall off the oplog. This solution would require a way to execute the filter only once, which could be implemented either by introducing a &lt;tt&gt;SegmentStage&lt;/tt&gt; to generate a sequence of incrementing integer values, or by adding an &quot;executeOnce&quot; mode to the existing &lt;tt&gt;FilterStage&lt;/tt&gt;. The issue caused by resolving the &lt;tt&gt;seekRecordId&lt;/tt&gt; before building the plan would require some further thought, possibly pushing down the logic to obtain the &lt;tt&gt;recordId&lt;/tt&gt; into the &lt;tt&gt;ScanStage&lt;/tt&gt; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b3c26f8c1403e499f4a996d684b298d3f3b1f12f/src/mongo/db/exec/collection_scan.cpp#L175-L186&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;as is in done in the classic engine&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Before committing to this work, however, we should confirm that the scenarios above can actually arise. This may involve adding failpoints into SBE plans to cause them to freeze or yield at the appropriate moment, forcing the oplog to rollover, then allowing the plan to continue.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1614256">SERVER-54350</key>
            <summary>Investigate potential race conditions in SBE oplog plans</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="andrew.paroski@mongodb.com">Drew Paroski</assignee>
                                    <reporter username="bernard.gorman@mongodb.com">Bernard Gorman</reporter>
                        <labels>
                            <label>post-rc0</label>
                            <label>sbe-post-rc0</label>
                    </labels>
                <created>Fri, 5 Feb 2021 16:56:42 +0000</created>
                <updated>Mon, 18 Oct 2021 16:48:29 +0000</updated>
                            <resolved>Tue, 1 Jun 2021 15:27:00 +0000</resolved>
                                                                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                    <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1766408">SERVER-57365</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1765027">SERVER-57325</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1454387">SERVER-50580</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 1 Jun 2021 15:25:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kyle.suarez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>
            <customfieldvalue>andrew.paroski@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hytl6n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyf4mf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4466">Query Execution 2021-03-22</customfieldvalue>
    <customfieldvalue id="4472">Query Execution 2021-05-03</customfieldvalue>
    <customfieldvalue id="4697">Query Execution 2021-05-17</customfieldvalue>
    <customfieldvalue id="4703">Query Execution 2021-05-31</customfieldvalue>
    <customfieldvalue id="4705">Query Execution 2021-06-14</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyt7fr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>