<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:07:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-67265] Errors reported from the `buildscripts_test` task are difficult to diagnose</title>
                <link>https://jira.mongodb.org/browse/SERVER-67265</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I made a change where I removed a .js test, but forgot to remove it from the relevant .yml files that were referencing it.&lt;/p&gt;

&lt;p&gt;When I ran Evergreen, I got two failed tasks - &lt;a href=&quot;https://spruce.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_required_buildscripts_test_patch_26a60b400fc8ceb021ddb81048a65850df4cb0e2_62a87b1a306615601643f916_22_06_14_12_12_28/tests?execution=0&amp;amp;sortBy=STATUS&amp;amp;sortDir=ASC&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;buildscripts_test&lt;/tt&gt;&lt;/a&gt; and &lt;a href=&quot;https://spruce.mongodb.com/task/mongodb_mongo_master_generate_tasks_for_version_version_gen_patch_26a60b400fc8ceb021ddb81048a65850df4cb0e2_62a87b1a306615601643f916_22_06_14_12_12_28/logs?execution=0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;version_gen&lt;/tt&gt;&lt;/a&gt;. The error in &lt;tt&gt;buildscripts_test&lt;/tt&gt; seems to clearly show me what the problem is. However, the one in &lt;a href=&quot;https://evergreen.mongodb.com/lobster/evergreen/task/mongodb_mongo_master_generate_tasks_for_version_version_gen_patch_26a60b400fc8ceb021ddb81048a65850df4cb0e2_62a87b1a306615601643f916_22_06_14_12_12_28/0/task#bookmarks=0%2C102%2C564%2C565%2C566%2C567%2C927%2C929%2C930%2C931%2C1150&amp;amp;f~=110~timestamp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;version_gen&lt;/tt&gt;&lt;/a&gt; is really hard to navigate and find and on top of that is related to that on &lt;tt&gt;version_gen&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;In the interest of developer productivity, this should be made more obvious to figure out.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2068274">SERVER-67265</key>
            <summary>Errors reported from the `buildscripts_test` task are difficult to diagnose</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="devprod-correctness-team@mongodb.com">[DO NOT ASSIGN] Backlog - DevProd Correctness</assignee>
                                    <reporter username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</reporter>
                        <labels>
                    </labels>
                <created>Tue, 14 Jun 2022 15:05:42 +0000</created>
                <updated>Mon, 23 Oct 2023 19:00:10 +0000</updated>
                                                                            <component>Testing Infrastructure</component>
                                        <votes>1</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="4679242" author="robert.guo" created="Thu, 14 Jul 2022 14:17:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=pierlauro.sciarelli%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;pierlauro.sciarelli@mongodb.com&quot;&gt;pierlauro.sciarelli@mongodb.com&lt;/a&gt; I think what Iryna was trying to say is that you can configure the Evergreen notifications &lt;a href=&quot;https://evergreen.mongodb.com/notifications&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;on this page&lt;/a&gt; so that you get notified when the first test fails. In this case, the failing test in &lt;tt&gt;buildscripts_test&lt;/tt&gt; should fail within a few minutes of &lt;tt&gt;version_gen&lt;/tt&gt; given neither depend on compile. Let me know if this strategy helps&lt;/p&gt;</comment>
                            <comment id="4679178" author="pierlauro.sciarelli" created="Thu, 14 Jul 2022 13:56:58 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=iryna.zhuravlova%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;iryna.zhuravlova@mongodb.com&quot;&gt;iryna.zhuravlova@mongodb.com&lt;/a&gt; that could be a way, but still sometimes it takes hours for a patch to start. And I still want to make the point that that there is no reason to make a whole patch fail in case a non-existing test file is excluded from some suite.&lt;/p&gt;</comment>
                            <comment id="4674118" author="JIRAUSER1257562" created="Tue, 12 Jul 2022 16:41:58 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=pierlauro.sciarelli%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;pierlauro.sciarelli@mongodb.com&quot;&gt;pierlauro.sciarelli@mongodb.com&lt;/a&gt;! Maybe subscribing for evergreen notifications might help in this case? This should tell you sooner if the patch build failed and get you a quick warning. Let us know if it won&apos;t.&#160;&#160;&lt;/p&gt;</comment>
                            <comment id="4617067" author="pierlauro.sciarelli" created="Wed, 15 Jun 2022 10:58:56 +0000"  >&lt;p&gt;Thanks for jumping on the issue &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robert.guo%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;robert.guo@mongodb.com&quot;&gt;robert.guo@mongodb.com&lt;/a&gt;!&lt;/p&gt;

&lt;p&gt;Sorry for making it not clear, my main observation was the following: having a non-existing test excluded from a suite should not be a severe error causing the failure of a whole patch. Right now, it is. This can affect developer productivity a lot: sometimes I send a patch and check it after 2 hours, just to discover that nothing has been executed because of this error. This effectively slows down development process.&lt;/p&gt;</comment>
                            <comment id="4616896" author="kaloian.manassiev" created="Wed, 15 Jun 2022 09:23:42 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robert.guo%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;robert.guo@mongodb.com&quot;&gt;robert.guo@mongodb.com&lt;/a&gt;, I meant that the output of &lt;a href=&quot;https://spruce.mongodb.com/task/mongodb_mongo_master_generate_tasks_for_version_version_gen_patch_26a60b400fc8ceb021ddb81048a65850df4cb0e2_62a87b1a306615601643f916_22_06_14_12_12_28/logs?execution=0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;version_gen&lt;/tt&gt;&lt;/a&gt; is very hard to follow, not &lt;tt&gt;buildscripts_test&lt;/tt&gt;. The latter actually gives me a meaningful error, but the former is very difficult to parse - I updated the description.&lt;/p&gt;</comment>
                            <comment id="4615697" author="robert.guo" created="Tue, 14 Jun 2022 19:38:08 +0000"  >&lt;p&gt;Hey Kal, I took another look at the failures and &lt;tt&gt;buildscripts_test&lt;/tt&gt; is in fact behaving as expected. The configuration in suite yamls is being presented and caught without having to wait for compile and the test to run. Regarding the less ergonomic &lt;tt&gt;version_gen&lt;/tt&gt; task failure; we&apos;re continuously integrating more of resmoke suites with Evergreen tasks to reduce the amount of dedicated CI logic (e.g. now you don&apos;t have to specify a suite option for synonymous Evergreen tasks). This means that when there&apos;s an issue with resmoke suites, task generation can&apos;t proceed. In general, it should be safe to shuttle &lt;tt&gt;version_gen&lt;/tt&gt; failures to Dev Prod, similar to failures in the old &lt;tt&gt;*_gen&lt;/tt&gt; tasks. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jeff.zambory%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;jeff.zambory@mongodb.com&quot;&gt;jeff.zambory@mongodb.com&lt;/a&gt; said DAG would be happy to make the error message in &lt;tt&gt;version_gen&lt;/tt&gt; nicer if you&apos;d prefer.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=pierlauro.sciarelli%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;pierlauro.sciarelli@mongodb.com&quot;&gt;pierlauro.sciarelli@mongodb.com&lt;/a&gt; regarding the 1h wait time; that&apos;s absolutely unacceptable. The current process I believe is to enable notifications for the first failure in Evergreen, which should in this case be either &lt;tt&gt;version_gen&lt;/tt&gt; or &lt;tt&gt;buildscripts_test&lt;/tt&gt;. Either should show up within a few minutes. Would this be an acceptable experience? More than happy to propose other solutions if not. The test in &lt;tt&gt;buildscripts_test&lt;/tt&gt; that failed should be exactly what you proposed. You can also run many checks locally with python (&lt;tt&gt;python -m unittest buildscripts.tests.my_test&lt;/tt&gt;). &lt;/p&gt;</comment>
                            <comment id="4614986" author="kaloian.manassiev" created="Tue, 14 Jun 2022 16:24:56 +0000"  >&lt;p&gt;Suggestion from &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=pierlauro.sciarelli%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;pierlauro.sciarelli@mongodb.com&quot;&gt;pierlauro.sciarelli@mongodb.com&lt;/a&gt; (from Slack):&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;What about proposing to simply add a task similar to the TODO check, that checks the presence of non-existing files in suite definitions?&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;And generates a ticket in case a problem is detected (edited) &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Suggestion from &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn@mongodb.com&quot;&gt;max.hirschhorn@mongodb.com&lt;/a&gt; (from Slack):&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;I&apos;d expect we&apos;d want a DAG ticket to fix mongodb/mongo-task-generator so it doesn&apos;t crash when there&apos;s something unexpected about the YAML suite file&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="4614924" author="pierlauro.sciarelli" created="Tue, 14 Jun 2022 16:08:19 +0000"  >&lt;p&gt;May I also argue that this kind of failure should not cause the crash of suites generation?&lt;/p&gt;

&lt;p&gt;If someone deletes &lt;tt&gt;jstests/XYZ.js&lt;/tt&gt; and this file happened to be in the exclude list of one suite, the generation of &lt;b&gt;ALL&lt;/b&gt; the suites fails. This is highly impacting developer productivity:&lt;br/&gt;
1. Schedule a patch&lt;br/&gt;
2. Wait one hour for the generation tasks to run&lt;br/&gt;
3. The whole patch fails&lt;/p&gt;

&lt;p&gt;I would propose to simply add an additional task to the variants that checks the presence of non-existing files in suite definitions and complains if one is detected.&lt;/p&gt;

&lt;p&gt;Right now this case is treated as a non-recoverable error and disallows executing any test.&lt;/p&gt;</comment>
                            <comment id="4614726" author="robert.guo" created="Tue, 14 Jun 2022 15:18:45 +0000"  >&lt;p&gt;Thanks for raising this issue Kal! I&apos;m assigning this to DAG given the error is from a task generation test. The error might seem obscure since it&apos;s in rust &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; &lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="26418"><![CDATA[Correctness]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 14 Jun 2022 15:18:45 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 29 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>iryna.zhuravlova@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 29 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>devprod-correctness-team@mongodb.com</customfieldvalue>
            <customfieldvalue>iryna.zhuravlova@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>pierlauro.sciarelli@mongodb.com</customfieldvalue>
            <customfieldvalue>robert.guo@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0yji7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0heq8:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0y5nj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>