<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:45:21 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-58750] Investigate whether InternalPlans::indexScan() executor always scans forward</title>
                <link>https://jira.mongodb.org/browse/SERVER-58750</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Investigate whether subsequent calls of &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ccba6ffd40461675c4a5558c7fceeeb996a85a10/src/mongo/db/query/plan_executor_impl.cpp#L306&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;getNextDocument&lt;/a&gt; can result in scanning a key lower than the previous one.&lt;/p&gt;

&lt;p&gt;The guess would be that this is not possible because index keys are sorted and - in case of yielding - it can at most happen for some previous key to be lost.&lt;/p&gt;

&lt;p&gt;Opening this ticket to double check because &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20700&quot; title=&quot;Invariant failure in collection_metadata.cpp during splitChunk command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20700&quot;&gt;&lt;del&gt;SERVER-20700&lt;/del&gt;&lt;/a&gt; introduced an additional sorting of the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1f4dd25739c182d7528847dd3303598a751d1390/src/mongo/db/s/split_vector.cpp#L348-L350&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;splitVector&lt;/a&gt; result (vector of keys filled in with subsequent &lt;tt&gt;getNextDocument&lt;/tt&gt; calls), so it would be worth checking if the original bug is still around.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1828216">SERVER-58750</key>
            <summary>Investigate whether InternalPlans::indexScan() executor always scans forward</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="pierlauro.sciarelli@mongodb.com">Pierlauro Sciarelli</assignee>
                                    <reporter username="pierlauro.sciarelli@mongodb.com">Pierlauro Sciarelli</reporter>
                        <labels>
                    </labels>
                <created>Thu, 22 Jul 2021 10:45:27 +0000</created>
                <updated>Sun, 29 Oct 2023 21:50:34 +0000</updated>
                            <resolved>Tue, 14 Dec 2021 13:43:00 +0000</resolved>
                                                    <fixVersion>5.3.0</fixVersion>
                                    <component>Query Execution</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="4246583" author="xgen-internal-githook" created="Tue, 14 Dec 2021 13:39:24 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-58750&quot; title=&quot;Investigate whether InternalPlans::indexScan() executor always scans forward&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-58750&quot;&gt;&lt;del&gt;SERVER-58750&lt;/del&gt;&lt;/a&gt; get rid of unnecessary sorting of split keys&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/bf7adca3bd56959f320eccb610668fe21c196243&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/bf7adca3bd56959f320eccb610668fe21c196243&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4036527" author="david.storch" created="Thu, 2 Sep 2021 14:54:11 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=pierlauro.sciarelli&quot; class=&quot;user-hover&quot; rel=&quot;pierlauro.sciarelli&quot;&gt;pierlauro.sciarelli&lt;/a&gt;! I have inspected the code more carefully, and I believe the &quot;always scans forward&quot; guarantee can indeed be relied upon. One important note is that this does not apply in general to any &lt;tt&gt;PlanExecutor&lt;/tt&gt;, or to any call to &lt;tt&gt;PlanExecutorImpl::getNextDocument()&lt;/tt&gt;. Instead, I confirmed that this guarantee exists specifically for &lt;tt&gt;PlanExecutor&lt;/tt&gt; objects that were constructed via the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d080cd863aeac79eacae88579749706d25842b1b/src/mongo/db/query/internal_plans.h#L98&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;InternalPlanner::indexScan()&lt;/tt&gt;&lt;/a&gt; interface. My understanding is that the relevant sharding code for splitVector is using &lt;tt&gt;InternalPlanner::indexScan()&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;I suspect that the original problem which spawned &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20700&quot; title=&quot;Invariant failure in collection_metadata.cpp during splitChunk command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20700&quot;&gt;&lt;del&gt;SERVER-20700&lt;/del&gt;&lt;/a&gt; was yielding-related, so I made sure to check that the yielding process could not cause a storage cursor to move backwards. During a yield, the underlying index cursor (a &lt;tt&gt;SortedDataInterface::Cursor&lt;/tt&gt;) is saved and restored, which causes the WT cursor to be unpositioned and then repositioned. The documentation for &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d080cd863aeac79eacae88579749706d25842b1b/src/mongo/db/storage/sorted_data_interface.h#L357&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;SortedDataInterface::restore()&lt;/tt&gt;&lt;/a&gt; states the following:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;If the former position no longer exists, a following call to next() will return the next closest position in the direction of the scan, if any.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;This sounds to me like it is guaranteeing that restore() cannot cause the position of the cursor to move backwards (or, more precisely, cannot cause the cursor to move opposite the direction of the scan). I double-checked the implementation of &lt;tt&gt;SortedDataInterface::Cursor&lt;/tt&gt; for WiredTiger, and it has explicit code to implement this behavior. See here: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d080cd863aeac79eacae88579749706d25842b1b/src/mongo/db/storage/wiredtiger/wiredtiger_index.cpp#L1065-L1103&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/d080cd863aeac79eacae88579749706d25842b1b/src/mongo/db/storage/wiredtiger/wiredtiger_index.cpp#L1065-L1103&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I am going to move this ticket to the sharding backlog for re-triage. Given the changes from commit &lt;a href=&quot;https://github.com/mongodb/mongo/commit/cdb43efd361b20a0c6a9bf7ad6e14dd307b6a1a4&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/cdb43efd361b20a0c6a9bf7ad6e14dd307b6a1a4&lt;/a&gt;&#160;against this ticket, it seems like this ticket can be repurposed for both deleting the TODO comments and removing the unnecessary sort of &lt;tt&gt;splitKeys&lt;/tt&gt;. If we simply close this ticket, then the &quot;TODO&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-58750&quot; title=&quot;Investigate whether InternalPlans::indexScan() executor always scans forward&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-58750&quot;&gt;&lt;del&gt;SERVER-58750&lt;/del&gt;&lt;/a&gt;&quot;&#160;comments added by commit cdb43efd36 will be orphaned.&lt;/p&gt;</comment>
                            <comment id="3957209" author="xgen-internal-githook" created="Thu, 22 Jul 2021 12:24:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-58750&quot; title=&quot;Investigate whether InternalPlans::indexScan() executor always scans forward&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-58750&quot;&gt;&lt;del&gt;SERVER-58750&lt;/del&gt;&lt;/a&gt; Add TODOs for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-58750&quot; title=&quot;Investigate whether InternalPlans::indexScan() executor always scans forward&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-58750&quot;&gt;&lt;del&gt;SERVER-58750&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/cdb43efd361b20a0c6a9bf7ad6e14dd307b6a1a4&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/cdb43efd361b20a0c6a9bf7ad6e14dd307b6a1a4&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 22 Jul 2021 12:24:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 8 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 8 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>pierlauro.sciarelli@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hztufb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hze4i7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4715">QE 2021-08-23</customfieldvalue>
    <customfieldvalue id="4717">QE 2021-09-06</customfieldvalue>
    <customfieldvalue id="5428">Sharding EMEA 2021-11-29</customfieldvalue>
    <customfieldvalue id="5429">Sharding EMEA 2021-12-13</customfieldvalue>
    <customfieldvalue id="5430">Sharding EMEA 2021-12-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hztgof:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>