<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:56:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-21030] Run changed/added tests 10x on patch builds</title>
                <link>https://jira.mongodb.org/browse/SERVER-21030</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Change to resmoke.py based on discussions with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mpobrien&quot; class=&quot;user-hover&quot; rel=&quot;mpobrien&quot;&gt;mpobrien&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dan%4010gen.com&quot; class=&quot;user-hover&quot; rel=&quot;dan@10gen.com&quot;&gt;dan@10gen.com&lt;/a&gt;.  The goal is that whenever an engineer runs a patch build that changes or adds a jstest, the patch build will automatically run with --repeat=10 for that test.&lt;/p&gt;</description>
                <environment></environment>
        <key id="235352">SERVER-21030</key>
            <summary>Run changed/added tests 10x on patch builds</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="michael.grundy">Michael Grundy</assignee>
                                    <reporter username="ian@mongodb.com">Ian Whalen</reporter>
                        <labels>
                    </labels>
                <created>Tue, 20 Oct 2015 16:25:20 +0000</created>
                <updated>Wed, 5 Apr 2017 16:44:21 +0000</updated>
                            <resolved>Fri, 17 Jun 2016 19:18:46 +0000</resolved>
                                                                    <component>Testing Infrastructure</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="1298341" author="kamran.khan" created="Fri, 17 Jun 2016 19:18:39 +0000"  >&lt;p&gt;This functionality was added as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-24537&quot; title=&quot;Create a new and modified test running task&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-24537&quot;&gt;&lt;del&gt;SERVER-24537&lt;/del&gt;&lt;/a&gt;.  We&apos;re only running with --repeat=2 for now, though.&lt;/p&gt;</comment>
                            <comment id="1110756" author="max.hirschhorn@10gen.com" created="Thu, 10 Dec 2015 15:40:16 +0000"  >&lt;p&gt;Filed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21841&quot; title=&quot;Add mode to resmoke.py to list under what suites a test runs&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21841&quot;&gt;&lt;del&gt;SERVER-21841&lt;/del&gt;&lt;/a&gt; to track a feature in resmoke.py to list the suites a test is run under. This can be used by an external tool to generate resmoke.py YAML configurations or invocations.&lt;/p&gt;

&lt;p&gt;Moving this ticket to the server tools team backlog because the remaining components would be (1) extracting the changed tests from the patch applied, (2) mapping resmoke.py&apos;s suites to Evergreen tasks from the &lt;tt&gt;etc/evergreen.yml&lt;/tt&gt; project configuration, and (3) generating resmoke.py invocations to run with &lt;tt&gt;&amp;#45;&amp;#45;repeat=10&lt;/tt&gt; as Charlie described.&lt;/p&gt;</comment>
                            <comment id="1067955" author="charlie.swanson" created="Thu, 22 Oct 2015 15:48:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=martin.bligh&quot; class=&quot;user-hover&quot; rel=&quot;martin.bligh&quot;&gt;martin.bligh&lt;/a&gt; what are you referring to by &apos;there&apos;? Could you move the discussion to the relevant ticket? I don&apos;t think this has much to do with this ticket anymore.&lt;/p&gt;</comment>
                            <comment id="1067952" author="martin.bligh" created="Thu, 22 Oct 2015 15:45:45 +0000"  >&lt;p&gt;Maybe we can just break out the logs there too?&lt;/p&gt;</comment>
                            <comment id="1067947" author="charlie.swanson" created="Thu, 22 Oct 2015 15:40:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=martin.bligh&quot; class=&quot;user-hover&quot; rel=&quot;martin.bligh&quot;&gt;martin.bligh&lt;/a&gt;&apos;s point about it being hard to read the logs locally is a good one. I have often run into this issue, where I run things in parallel to test my changes, have something fail, and then have to re-run just that one test to figure out why it&apos;s failing. I&apos;ve filed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21073&quot; title=&quot;Add option to resmoke.py to automatically re-run tests failed tests when running with multiple jobs&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21073&quot;&gt;&lt;del&gt;SERVER-21073&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21074&quot; title=&quot;Add ability to resmoke.py to pick up where you left off running a suite&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21074&quot;&gt;&lt;del&gt;SERVER-21074&lt;/del&gt;&lt;/a&gt; to improve the workflow of using many jobs to run a large suite locally. We had already discussed the latter, but I couldn&apos;t find a ticket, so I created one.&lt;/p&gt;</comment>
                            <comment id="1067856" author="martin.bligh" created="Thu, 22 Oct 2015 14:44:41 +0000"  >&lt;p&gt;Ah, OK, I was looking at local logs - good point&lt;/p&gt;</comment>
                            <comment id="1067854" author="milkie" created="Thu, 22 Oct 2015 14:43:33 +0000"  >&lt;p&gt;But it&apos;s not like compilation with -j, because each parallel stream keeps a separate log, which is accessible via the Evergreen UI.  (It&apos;s true that if you make use of -j parallelism to run tests with resmoke on your local machine, the logs by default all go to the same place and interleave.)&lt;/p&gt;</comment>
                            <comment id="1067850" author="martin.bligh" created="Thu, 22 Oct 2015 14:40:17 +0000"  >&lt;p&gt;FWIW, now that we started running a lot of tests in parallel, it seems like it&apos;s going to be a lot more difficult to see the error messages from a failing test (just like compilation with -j) ... not sure if it&apos;s also possible to repeat the test(s) that failed single threaded at the end. &lt;/p&gt;

&lt;p&gt;I don&apos;t think that ties into this ticket, since I suspect you have a lot more context inside resmoke already at this point, but thought I&apos;d mention it now in case.&lt;/p&gt;</comment>
                            <comment id="1067834" author="milkie" created="Thu, 22 Oct 2015 14:31:34 +0000"  >&lt;p&gt;We cannot derive the test suites from the relative path of any .js file because any jstest file can be referenced by any suite in any .yml config file (which is how it should be).&lt;br/&gt;
As Charlie said, we will need to scan all the suites to see which ones contain the changed jstest files, and then run &quot;something&quot; that only tests the changed files in each suite.  However, I don&apos;t think the Evergreen interface is prepared for this; there may be a lot of work to do to make the output from this &quot;fit&quot; into the current UI for a patch build and still have all the links to logs and error detection work sanely.&lt;/p&gt;</comment>
                            <comment id="1067806" author="martin.bligh" created="Thu, 22 Oct 2015 14:08:38 +0000"  >&lt;p&gt;Can we derive the tests suites from the relative path of the .js file in an automated fashion? &lt;br/&gt;
It doesn&apos;t seem like the directory names, the evergreen test names, and the test suite names are consistent, which is unfortunate - makes it harder for manual repro too.&lt;br/&gt;
Maybe we can come up with a rule? I presume it&apos;s 1-to-many from dir name to suite name though&lt;/p&gt;</comment>
                            <comment id="1067792" author="mpobrien" created="Thu, 22 Oct 2015 13:52:49 +0000"  >&lt;blockquote&gt;&lt;p&gt;What sort of changes to resmoke.py are we envisioning? Actually doing the git diff to see which tests changed? That seems like an odd responsibility for resmoke.py to have.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Logic for this would definitely be external, it would be gnarly for resmoke to have to touch anything involving git.&lt;br/&gt;
We were basically just looking for the ability to override the &quot;repeat&quot; flag for particular tests only. If the behavior can already be accomplished using what you stated in #3 then we&apos;re already good here&lt;/p&gt;</comment>
                            <comment id="1067545" author="charlie.swanson" created="Thu, 22 Oct 2015 02:44:50 +0000"  >&lt;p&gt;What sort of changes to resmoke.py are we envisioning? Actually doing the git diff to see which tests changed? That seems like an odd responsibility for resmoke.py to have.&lt;/p&gt;

&lt;p&gt;I don&apos;t think it would be hard for some logic external to resmoke.py to figure it out. Either evergreen could do it, or we could write some fun bash script in our evergreen.yml to do something like the following before or after running any given suite:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Look at the patch/diff to see which test files are affected&lt;/li&gt;
	&lt;li&gt;List the tests in that suite to check if the affected test(s) is/are part of the suite, something like
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;python buildscripts/resmoke.py --suite=&amp;lt;suite&amp;gt; -n | grep path/to/changed/test.js&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;for each test&lt;/p&gt;&lt;/li&gt;
	&lt;li&gt;If the test(s) is/are part of the suite, it could run the following after the suite to accomplish the desired behavior:
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;python buildscripts/resmoke.py --executor=&amp;lt;suite&amp;gt; path/to/changed/test1.js path/to/changed/test2.js --repeat=10 --continueOnFailure &amp;lt;other options, e.g. buildlogger setup&amp;gt;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="1067504" author="ian@10gen.com" created="Thu, 22 Oct 2015 00:40:14 +0000"  >&lt;p&gt;Pretty certain that the goal is to make it a bit more intelligent than that and only run the specific jstest files 10x.&lt;/p&gt;</comment>
                            <comment id="1065883" author="milkie" created="Tue, 20 Oct 2015 17:07:29 +0000"  >&lt;p&gt;Since patch builds run test suites, not individual jstest files, did you mean to say that the entire suite will run with --repeat=10, or is this more involved than that?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="243224">SERVER-21841</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="293501">SERVER-24537</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 20 Oct 2015 17:07:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 34 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-21841'>SERVER-21841</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-565</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 34 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>kamran.khan</customfieldvalue>
            <customfieldvalue>martin.bligh</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
            <customfieldvalue>michael.grundy</customfieldvalue>
            <customfieldvalue>mikeo@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkr9j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrdb3r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="903">TIG 12 (04/01/16)</customfieldvalue>
    <customfieldvalue id="1004">TIG 16 (06/24/16)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11861" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>User Summary</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11856"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkzuf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>