<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:43:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-80277] [CQF] Make plan selection as deterministic (repeatable) as possible</title>
                <link>https://jira.mongodb.org/browse/SERVER-80277</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Filing this ticket to capture some ideas discussed at the Bonsai engineering sync. This ticket contains many ideas and I fully expect this ticket to split off into several tickets (or its own PM) in the future.&lt;/p&gt;

&lt;p&gt;The consensus of the meeting was that it seems fairly important to do our best to ensure that the same query run multiple times results in the same plan being chosen. This can help customers and support teams find issues faster; the classic engine has the same problem.&lt;/p&gt;

&lt;p&gt;The main way this can manifest in Bonsai is when two plans have identical costs according to the cost model and we pick the one explored first. If two rewrites are added to the queue with equal priority (and result in identical costs), they may be explored in a different order, resulting in a different plan being selected on subsequent runs of the same query. There are also other instances in the optimizer that use hash-based containers that do no guarantee stable iteration order that can result in similar outcomes.&lt;/p&gt;

&lt;p&gt;Some potential solutions discussed:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Deterministic cost tie-breaking by using a lexicographical ordering of the plan itself.&lt;/li&gt;
	&lt;li&gt;Assign/enforce unique priorities to all rewrites so that they are always performed in the same order.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The other way this problem can manifest is through changes in CE, especially sampling-based CE. Using random sampling of the collection can result in different CE estimates on the same predicates.&lt;/p&gt;

&lt;p&gt;Some potential solutions discussed:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Use sequential sampling (read from the start of the collection)&lt;/li&gt;
	&lt;li&gt;Start the random cursor used for sampling with the same seed on each run&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Lastly, this problem affects our testing strategy for bonsai. The most common unit test will construct some input logical ABT, invoke the optimizer, and assert on the outputted physical ABT. The order of rewrites which are invoked changes the projection names which are generated, meaning that these tests can non-deterministically fail due to equivalent plans but different projection names.&lt;/p&gt;

&lt;p&gt;Some potential solutions discussed:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Change tests to only assert on the structure that they want to verify&lt;/li&gt;
	&lt;li&gt;Perform some plan canonicalization which will deterministically rename projections before asserting on plan explain equality&lt;/li&gt;
	&lt;li&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;Overlapping with ideas above&amp;#93;&lt;/span&gt; Ensure that rewrites are applied in consistent order&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2422490">SERVER-80277</key>
            <summary>[CQF] Make plan selection as deterministic (repeatable) as possible</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="ben.shteinfeld@mongodb.com">Ben Shteinfeld</reporter>
                        <labels>
                    </labels>
                <created>Mon, 21 Aug 2023 18:13:03 +0000</created>
                <updated>Thu, 21 Sep 2023 19:07:48 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                    <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2420064">SERVER-80202</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2435510">SERVER-80808</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2441681">SERVER-81052</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 21 Aug 2023 18:17:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        24 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3240</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>hana.pearlman@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            24 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>ben.shteinfeld@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2n5zj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i257eg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2ms4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>