<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:46:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37679] Concurrent sharded fuzzers should not run mapReduces with replace</title>
                <link>https://jira.mongodb.org/browse/SERVER-37679</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Concurrent mapReduces with replace are not supported. As part of the cluster&apos;s map reduce logic, we will always drop the output collection then shard it again. Since these two operations are not atomic, it can leads to a race condition where two mapReduces can simultaneously output to the same collection with the same UUID. &lt;/p&gt;

&lt;p&gt; With mapReduce A and B, in order:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;mapReduce A drops the output collection&lt;/li&gt;
	&lt;li&gt;mapReduce B drops the output collection&lt;/li&gt;
	&lt;li&gt;mapReduce A creates the sharded output collection with a generated UUID&lt;/li&gt;
	&lt;li&gt;mapReduce B attempts to shard the output collection, but when doing is told that the collection is already sharded and receives the UUID that mapReduce A generated.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;In order to avoid this undefined behavior, no sharded fuzzer suites (specifically: jstestfuzz_sharded_causal_consistency) should run mapReduce with replace.&lt;/p&gt;
</description>
                <environment></environment>
        <key id="621887">SERVER-37679</key>
            <summary>Concurrent sharded fuzzers should not run mapReduces with replace</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="blake.oler@mongodb.com">Blake Oler</assignee>
                                    <reporter username="blake.oler@mongodb.com">Blake Oler</reporter>
                        <labels>
                    </labels>
                <created>Fri, 19 Oct 2018 20:50:24 +0000</created>
                <updated>Fri, 22 Mar 2019 13:34:41 +0000</updated>
                            <resolved>Fri, 22 Mar 2019 13:34:41 +0000</resolved>
                                    <version>3.6.8</version>
                    <version>4.0.3</version>
                    <version>4.1.4</version>
                                                    <component>MapReduce</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="2188495" author="blake.oler" created="Fri, 22 Mar 2019 13:34:41 +0000"  >&lt;p&gt;Closing as Won&apos;t Fix. The issue has been prevented from happening in our testing infrastructure  and a DOCS ticket has been filed to warn users not to run mapReduces in this fashion.&lt;/p&gt;</comment>
                            <comment id="2153294" author="kaloian.manassiev" created="Fri, 15 Feb 2019 15:44:48 +0000"  >&lt;p&gt;Assigning to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=blake.oler&quot; class=&quot;user-hover&quot; rel=&quot;blake.oler&quot;&gt;blake.oler&lt;/a&gt; to file a DOCS ticket with what needs to be updated in the documentation and close this ticket, possibly black-listing the failing suites.&lt;/p&gt;</comment>
                            <comment id="2141872" author="alyson.cabral" created="Thu, 7 Feb 2019 23:02:51 +0000"  >&lt;p&gt;This should be documented. &lt;/p&gt;</comment>
                            <comment id="2141321" author="esha.maharishi@10gen.com" created="Thu, 7 Feb 2019 17:43:28 +0000"  >&lt;p&gt;Note that we can&apos;t really &quot;ban&quot; the concurrent mapReduces without just doing the fix (holding the distlock across the dropCollection and shardCollection), since the mapReduces can be sent to different routers.&lt;/p&gt;</comment>
                            <comment id="2118822" author="asya" created="Thu, 17 Jan 2019 20:39:40 +0000"  >&lt;p&gt;Should this limitation be in our documentation?&lt;/p&gt;</comment>
                            <comment id="2074839" author="esha.maharishi@10gen.com" created="Thu, 29 Nov 2018 01:50:07 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt;, I believe the mapReduce will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/s/commands/cluster_map_reduce_cmd.cpp#L270-L276&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;fail early on mongos&lt;/a&gt;&#160;if &apos;out&apos; is specified without &apos;sharded: true&apos; but mongos believes the collection is sharded. A test for this was added recently &lt;a href=&quot;https://github.com/mongodb/mongo/commit/7dbcd710077bc4141e71730be9e12558880375e6#diff-2e9f3ac3a4c2da7a35b07390d01d90bdR36&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I think the fact that mongos may be stale won&apos;t be an issue, since the fuzzer suites use only one mongos.&lt;/p&gt;</comment>
                            <comment id="2074835" author="max.hirschhorn@10gen.com" created="Thu, 29 Nov 2018 01:42:22 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=esha.maharishi&quot; class=&quot;user-hover&quot; rel=&quot;esha.maharishi&quot;&gt;esha.maharishi&lt;/a&gt;, the (concurrent) fuzzer cannot know if the collection is sharded when it goes to run the &lt;tt&gt;mapReduce&lt;/tt&gt; command.&lt;/p&gt;</comment>
                            <comment id="2074778" author="esha.maharishi@10gen.com" created="Wed, 28 Nov 2018 23:52:09 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt;, we would ban mapReduces with &lt;b&gt;output&lt;/b&gt;&#160;to a &lt;b&gt;sharded&lt;/b&gt;&#160;collection. We could still leave in mapReduces with sharded &lt;b&gt;input&lt;/b&gt; and with output to an &lt;b&gt;unsharded&lt;/b&gt;&#160;collection with any of replace, merge, or reduce modes.&lt;/p&gt;</comment>
                            <comment id="2045407" author="esha.maharishi@10gen.com" created="Mon, 29 Oct 2018 19:26:32 +0000"  >&lt;p&gt;Note, we may have to blacklist mapReduce with &lt;b&gt;reduce&lt;/b&gt; and &lt;b&gt;merge&lt;/b&gt;&#160;(not just &lt;b&gt;replace&lt;/b&gt;) to an output collection as well, since &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.1.3/src/mongo/s/commands/cluster_map_reduce_cmd.cpp#L474-L479&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;if the output collection does not exist or is empty, both of these modes drop and re-shard the output collection&lt;/a&gt;, which leads to the same race.&lt;/p&gt;

&lt;p&gt;I saw this in a BF in a sharded concurrent fuzzer recently and hadn&apos;t finished investigating it, but that was the likely cause.&lt;/p&gt;</comment>
                            <comment id="2036630" author="esha.maharishi@10gen.com" created="Fri, 19 Oct 2018 22:18:39 +0000"  >&lt;p&gt;On further thought, we may be able to solve this by creating an atomic &quot;dropAndShardCollection&quot; command that holds distlocks across the drop and shardCollection operations.&lt;/p&gt;

&lt;p&gt;However, I am not sure we want to invest further time into sharded mapReduce with replace, so blacklisting mapReduce with replace from the sharded concurrent fuzzers may be the only alternative.&lt;/p&gt;</comment>
                            <comment id="2036520" author="blake.oler" created="Fri, 19 Oct 2018 20:50:39 +0000"  >&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=esha.maharishi&quot; class=&quot;user-hover&quot; rel=&quot;esha.maharishi&quot;&gt;esha.maharishi&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                        <issuelink>
            <issuekey id="721075">DOCS-12566</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="647740">SERVER-38471</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="647740">SERVER-38471</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 19 Oct 2018 20:52:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 46 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>blake.oler@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 46 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>53.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alyson.cabral@mongodb.com</customfieldvalue>
            <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>blake.oler@mongodb.com</customfieldvalue>
            <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huauiv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu153z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2787">Sharding 2019-02-25</customfieldvalue>
    <customfieldvalue id="2824">Sharding 2019-03-11</customfieldvalue>
    <customfieldvalue id="2825">Sharding 2019-03-25</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huags7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>