<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:06:49 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-66926] Introduce end-to-end functional query correctness testing based on golden test framework</title>
                <link>https://jira.mongodb.org/browse/SERVER-66926</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Add a new suite called something like &lt;tt&gt;query_integration_tests_standalone&lt;/tt&gt;. This would work much like the pre-existing &lt;tt&gt;integration_tests_standalone&lt;/tt&gt;, but it would be focused on testing the correctness of query results or potentially even testing plans using explain. Critically, the idea is to use golden file based testing &amp;#8211; the tests would run some queries or explain operations, dump their results, and then compare the results to the golden files. Having an integration test suite like this is an important part of our testing strategy for the Common Query Framework. It could also strengthen our query correctness testing in general &amp;#8211; the current testing is either based on the generational fuzzer which may not generate realistic queries, or based on making specific assertions about the nature of the query results which are weaker tests.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2059709">SERVER-66926</key>
            <summary>Introduce end-to-end functional query correctness testing based on golden test framework</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="david.storch@mongodb.com">David Storch</reporter>
                        <labels>
                    </labels>
                <created>Wed, 1 Jun 2022 15:41:26 +0000</created>
                <updated>Tue, 6 Dec 2022 00:04:02 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="4659148" author="david.percy" created="Wed, 6 Jul 2022 00:17:32 +0000"  >&lt;blockquote&gt;
&lt;p&gt;David Percy, does &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67415&quot; title=&quot;Expose golden-test framework to jstests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67415&quot;&gt;&lt;del&gt;SERVER-67415&lt;/del&gt;&lt;/a&gt; include introducing a new test suite which is supposed to contain only tests which use the golden framework (in JS) to check query correctness?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Yes, I&apos;m planning to add a new suite for these tests.  Maybe jstests/query_golden/.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I do believe that we should isolate the golden JS tests into such a suite to avoid the perils that passthrough testing poses to a golden data based approach.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I agree we shouldn&apos;t enable any sharding/repl passthroughs yet.  But I think we should at least run these new tests against both classic and CQF: running against classic helps us make sure the tests / expected output are correct.&lt;/p&gt;

&lt;p&gt;It is important though for each test author to avoid making brittle assertions, like assuming an order for find() without a sort().  &lt;/p&gt;</comment>
                            <comment id="4636715" author="david.storch" created="Thu, 23 Jun 2022 21:14:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.percy%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;david.percy@mongodb.com&quot;&gt;david.percy@mongodb.com&lt;/a&gt; is working on related ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67415&quot; title=&quot;Expose golden-test framework to jstests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67415&quot;&gt;&lt;del&gt;SERVER-67415&lt;/del&gt;&lt;/a&gt;, so I don&apos;t think there is an immediate need for this ticket. The idea of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67415&quot; title=&quot;Expose golden-test framework to jstests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67415&quot;&gt;&lt;del&gt;SERVER-67415&lt;/del&gt;&lt;/a&gt; is similar, but allows the test author to write JS executed in the &lt;tt&gt;mongo&lt;/tt&gt; shell rather than writing a C++ integration test. I&apos;m going to move this ticket into the triage queue since I&apos;m not entirely sure where it should live.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.percy%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;david.percy@mongodb.com&quot;&gt;david.percy@mongodb.com&lt;/a&gt;, does &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67415&quot; title=&quot;Expose golden-test framework to jstests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67415&quot;&gt;&lt;del&gt;SERVER-67415&lt;/del&gt;&lt;/a&gt; include introducing a new test suite which is supposed to contain only tests which use the golden framework (in JS) to check query correctness? I do believe that we should isolate the golden JS tests into such a suite to avoid the perils that passthrough testing poses to a golden data based approach.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2073496">SERVER-67415</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 6 Jul 2022 00:17:32 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 31 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 31 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>david.percy@mongodb.com</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0x2rr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i00rj0:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5896">QE 2022-06-13</customfieldvalue>
    <customfieldvalue id="5898">QE 2022-06-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0wox3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>