<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:11:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-26083] create library of basic ops that can be run against a cluster under different conditions</title>
                <link>https://jira.mongodb.org/browse/SERVER-26083</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This is similar in nature to two things we do already:&lt;/p&gt;

&lt;p&gt;1) the concurrency suite&lt;br/&gt;
2) passthrough suites, including the stepdown and last-stable suites&lt;/p&gt;

&lt;p&gt;However, 1) doesn&apos;t give control over what is being run concurrently with the ops, and 2) is a heavyweight approach that requires blacklisting existing tests that don&apos;t work with the passthrough and considering whether each new test should be blacklisted.&lt;/p&gt;

&lt;p&gt;I&apos;d instead like a simple js lib containing something like:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testCRUD = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testAgg = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testCount = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testDistinct = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testGroup = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testFindAndModify = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(coll) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; testMapReduce = &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;function&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(db) { ... }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;which can be load()&apos;ed in a particular jstest. The jstest can then configure the cluster in any way desired (take down some nodes, slow down replication, drop messages, exercise the balancer, delete administrative sharding docs (e.g. shardIdentity, cluster version)) or the collection in any way desired (sharded or unsharded, with or without particular indexes) before running any subset (or all) of the functions in the lib.&lt;/p&gt;

&lt;p&gt;This would be useful because we re-write these kinds of functions often in many jstests, which is time-consuming and more likely to have bugs.&lt;/p&gt;

&lt;p&gt;A nice example the library could be based on is &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.3.12/jstests/sharding/collation_targeting.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.3.12/jstests/sharding/collation_targeting.js&lt;/a&gt;, which &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; wrote for 3.4.&lt;/p&gt;</description>
                <environment></environment>
        <key id="315855">SERVER-26083</key>
            <summary>create library of basic ops that can be run against a cluster under different conditions</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="backlog-server-tig">DO NOT USE - Backlog - Test Infrastructure Group (TIG)</assignee>
                                    <reporter username="esha.maharishi@mongodb.com">Esha Maharishi</reporter>
                        <labels>
                    </labels>
                <created>Tue, 13 Sep 2016 12:23:25 +0000</created>
                <updated>Wed, 23 Nov 2016 16:36:34 +0000</updated>
                            <resolved>Wed, 23 Nov 2016 16:36:34 +0000</resolved>
                                                                    <component>Testing Infrastructure</component>
                                        <votes>1</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1441362" author="max.hirschhorn@10gen.com" created="Wed, 23 Nov 2016 16:36:34 +0000"  >&lt;p&gt;To summarize my viewpoint: I think using a whitelist for choosing what operations are executed against a cluster can lead to scenarios where new functionality doesn&apos;t get tested under all configurations we&apos;re interested in testing. I would rather focus our efforts in making the blacklisting process easier if that is where the most pain with the current workflow exists. However, it didn&apos;t seem like we were able to reach an understanding on what improvements to blacklisting would help you accomplish the same goals. I&apos;m closing this ticket as &quot;Won&apos;t fix&quot;, but if there are ideas for making the blacklisting process easier or answers regarding the questions I asked previously, then we can revisit this ticket or address them under a new one.&lt;/p&gt;</comment>
                            <comment id="1439193" author="max.hirschhorn@10gen.com" created="Mon, 21 Nov 2016 17:23:33 +0000"  >&lt;blockquote&gt;
&lt;p&gt;What I&apos;d like is really the inverse: to &lt;b&gt;add new cluster configurations&lt;/b&gt; and immediately test them over a pre-defined set of features/functionality.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I&apos;m failing to grasp the benefit of having a pre-defined set of features/functionality continuously being tested in Evergreen other than &quot;blacklisting requires effort&quot;; I think we&apos;d be doing our future selves a disservice if we added tests for different cluster configurations and didn&apos;t automatically test new features/functionality under those cluster configurations.&lt;/p&gt;

&lt;p&gt;Would having a pre-defined set of features/functionality to test against the new cluster configuration be useful for &lt;em&gt;prototyping&lt;/em&gt; the new cluster configuration?&lt;/p&gt;

&lt;p&gt;I&apos;d appreciate it if you would answer the specific questions I asked &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-26083?focusedCommentId=1434144&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1434144&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;in my previous comment&lt;/a&gt; to help me better understand what other possible changes we could make to advance towards the same goal.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Currently we do it by adding a new jstest with a cluster configured the new way and manually re-writing (or copy-pasting from other tests) code to run a set of desired basic ops under that configuration.&lt;/p&gt;

&lt;p&gt;Some examples are:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster_initializes_sharding_awareness.js#L14-L27&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster_initializes_sharding_awareness.js#L14-L27&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster.js#L18-L31&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster.js#L18-L31&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/mixed_storage_version_replication.js#L17-L464&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/mixed_storage_version_replication.js#L17-L464&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;

&lt;p&gt;Does the &lt;tt&gt;testCRUD()&lt;/tt&gt; function in the &lt;tt&gt;upgrade_cluster*.js&lt;/tt&gt; tests act as anything more than a basic sanity check? Is there anything specific about the multiversion tests that makes blacklisting tests from the &lt;tt&gt;jstests/core/&lt;/tt&gt; directory or elsewhere more difficult?&lt;/p&gt;

&lt;p&gt;I should also mention that without being able to transfer ownership of the cluster (a potential outcome of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21774&quot; title=&quot;Determine Fixture Ownership for Our Tests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21774&quot;&gt;&lt;del&gt;SERVER-21774&lt;/del&gt;&lt;/a&gt;) and teaching resmoke.py how to run older version of MongoDB (a potential side-benefit of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21774&quot; title=&quot;Determine Fixture Ownership for Our Tests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21774&quot;&gt;&lt;del&gt;SERVER-21774&lt;/del&gt;&lt;/a&gt; depending on the approach taken), it isn&apos;t possible to write a resmoke.py suite that launches a mixed-version cluster and runs existing tests against it. I think it is likely more work for the TIG team to enable writing tests in that fashion; however, I think it worthwhile in its own right and would also bring us to a more consistent state with other resmoke.py suites in terms of how they are written and their reporting in Evergreen.&lt;/p&gt;</comment>
                            <comment id="1434662" author="esha.maharishi@10gen.com" created="Tue, 15 Nov 2016 17:09:49 +0000"  >&lt;p&gt;These are good points. I think the crux of the issue is:&lt;/p&gt;

&lt;p&gt;The jstests/core directory allows us to &lt;b&gt;add new features/functionality to mongod&lt;/b&gt; and immediately test them under a pre-defined set of cluster configurations, which are defined by the suites that run jstests/core. (We also do something similar with the jstests/sharding directory, with the continuous_stepdown and last_stable suites).&lt;/p&gt;

&lt;p&gt;What I&apos;d like is really the inverse: to &lt;b&gt;add new cluster configurations&lt;/b&gt; and immediately test them over a pre-defined set of features/functionality.&lt;/p&gt;

&lt;p&gt;We could arguably do this by adding a new suite for each new configuration, but it&apos;s difficult to stabilize a new suite, because it requires weeding out and blacklisting all tests that aren&apos;t compatible for one reason or another.&lt;/p&gt;

&lt;p&gt;Currently we do it by adding a new jstest with a cluster configured the new way and manually re-writing (or copy-pasting from other tests) code to run a set of desired basic ops under that configuration.&lt;/p&gt;

&lt;p&gt;Some examples are:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster_initializes_sharding_awareness.js#L14-L27&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster_initializes_sharding_awareness.js#L14-L27&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster.js#L18-L31&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/upgrade_cluster.js#L18-L31&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/mixed_storage_version_replication.js#L17-L464&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/multiVersion/mixed_storage_version_replication.js#L17-L464&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1434144" author="max.hirschhorn@10gen.com" created="Tue, 15 Nov 2016 06:34:43 +0000"  >&lt;blockquote&gt;
&lt;p&gt;This is similar in nature to two things we do already:&lt;/p&gt;

&lt;p&gt;1) the concurrency suite&lt;br/&gt;
2) passthrough suites, including the stepdown and last-stable suites&lt;/p&gt;

&lt;p&gt;However, 1) doesn&apos;t give control over what is being run concurrently with the ops, and 2) is a heavyweight approach that requires blacklisting existing tests that don&apos;t work with the passthrough and considering whether each new test should be blacklisted.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;FWIW, #1 also requires blacklisting tests - both in FSM runners such as &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/concurrency/fsm_all_sharded_replication.js#L8-L93&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;fsm_all_sharded_replication.js&lt;/tt&gt;&lt;/a&gt; and in other users of FSM workloads such as &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/jstests/noPassthrough/backup_restore.js#L67-L71&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;backup_restore.js&lt;/tt&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I&apos;m not sure I understand what control you want to have over the operations being run in the concurrency suite. Is it about exposing more configuration options to the cluster started by the concurrency framework?&lt;/p&gt;

&lt;p&gt;As of the writing of this comment, we have over 15 suites that reuse tests from the &lt;tt&gt;jstests/core/&lt;/tt&gt; directory. This means that there is a large force multiplier any time a developer adds a new test to the &lt;tt&gt;jstests/core/&lt;/tt&gt; directory beause the test will automatically be run under many different configurations. Each new configuration we add to reuse our existing tests strengthens the benefit of doing so. Additionally, having new tests automatically run under these different configurations means that new features and/or changes in behaviors are automatically tested under those configurations. I&apos;m reluctant to introduce yet another place to define test cases for building up more complex test suites.&lt;/p&gt;

&lt;p&gt;To the other part of your remark, enabling a new suite requires effort and given that there may be implicit assumptions in the tests about the cluster&apos;s configuration or regarding the consistency in behavior of mongod and mongos, I don&apos;t imagine that ever going away entirely. Perhaps some of that work could be lessened if we attempted to use tags for maintaining the blacklist of a resmoke.py suite; for example, the blacklists of the &lt;tt&gt;sharding_jscore_passthrough&lt;/tt&gt; suite and the &lt;tt&gt;sharded_collections_jscore_passthrough&lt;/tt&gt; suite could be heavily consolidated if we did &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18395&quot; title=&quot;Convert sharding_jscore_passthrough suite to use tag-based blacklist&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18395&quot;&gt;&lt;del&gt;SERVER-18395&lt;/del&gt;&lt;/a&gt;. Would making blacklisting tests easier address your motivation for filing this ticket?&lt;/p&gt;

&lt;p&gt;If not, are you dissatisfied by the quality of the tests in the &lt;tt&gt;jstests/core/&lt;/tt&gt; directory? Or other tests that we reuse for that matter?&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The jstest can then configure the cluster in any way desired (take down some nodes, slow down replication, drop messages, exercise the balancer, delete administrative sharding docs (e.g. shardIdentity, cluster version)) or the collection in any way desired (sharded or unsharded, with or without particular indexes) before running any subset (or all) of the functions in the lib.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;What would you say prevents such a suite from being written today by defining a new resmoke.py suite and reusing existing tests in the &lt;tt&gt;jstests/core/&lt;/tt&gt; directory, FSM workloads in the &lt;tt&gt;jstests/concurrency/fsm_workloads/&lt;/tt&gt; directory, etc.?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="314382">SERVER-25971</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 15 Nov 2016 06:34:43 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 12 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 12 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-tig</customfieldvalue>
            <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjw9b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hspozj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1314">Sharding 2016-11-21</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs624v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>