<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:49:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38480] Make Map-Reduce fully interruptible</title>
                <link>https://jira.mongodb.org/browse/SERVER-38480</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We disallow interruptions in Map-Reduce &lt;a href=&quot;https://github.com/mongodb/mongo/blob/99ec5dabaf1286a5440da0bb561e32d3aa79466a/src/mongo/db/commands/mr.cpp#L1397&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;on single node&lt;/a&gt;&#160;and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/99ec5dabaf1286a5440da0bb561e32d3aa79466a/src/mongo/db/commands/mr.cpp#L1704&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;on shards&lt;/a&gt;. They will conflict with prepared transactions on stepdown and shutdown. We can either make&#160;Map-Reduce interruptible or use weaker IX and IS locks instead.&#160;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/99ec5dabaf1286a5440da0bb561e32d3aa79466a/src/mongo/db/commands/mr.cpp#L446&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;dropTempCollections()&lt;/a&gt; is also protected by&#160;UninterruptibleLockGuard. If the temp collections are only in &lt;tt&gt;local&lt;/tt&gt; database as done by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35365&quot; title=&quot;MapReduce temporary inc collections should be written to the local database&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35365&quot;&gt;&lt;del&gt;SERVER-35365&lt;/del&gt;&lt;/a&gt;, they won&apos;t conflict with prepared transactions. This ticket should also audit that.&lt;/p&gt;</description>
                <environment></environment>
        <key id="647792">SERVER-38480</key>
            <summary>Make Map-Reduce fully interruptible</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="justin.seyster@mongodb.com">Justin Seyster</assignee>
                                    <reporter username="siyuan.zhou@mongodb.com">Siyuan Zhou</reporter>
                        <labels>
                            <label>prepare_interruptibility</label>
                    </labels>
                <created>Fri, 7 Dec 2018 23:06:37 +0000</created>
                <updated>Sun, 29 Oct 2023 22:25:58 +0000</updated>
                            <resolved>Thu, 31 Jan 2019 21:03:45 +0000</resolved>
                                                    <fixVersion>4.1.8</fixVersion>
                                    <component>MapReduce</component>
                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="2134003" author="xgen-internal-githook" created="Thu, 31 Jan 2019 21:03:15 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Justin Seyster&apos;, &apos;email&apos;: &apos;justin.seyster@mongodb.com&apos;, &apos;username&apos;: &apos;jseyster&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38480&quot; title=&quot;Make Map-Reduce fully interruptible&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38480&quot;&gt;&lt;del&gt;SERVER-38480&lt;/del&gt;&lt;/a&gt; Make Map-Reduce fully interruptible&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/57b22a11d206272a78124ee03c5a6cf26b3e1105&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/57b22a11d206272a78124ee03c5a6cf26b3e1105&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2103912" author="justin.seyster" created="Thu, 3 Jan 2019 00:16:11 +0000"  >&lt;p&gt;I spent some time inspecting the MapReduceCommand and MapReduceFinishCommand run methods, and except for two easily fixable ON_BLOCK_EXIT, I don&apos;t see any destructor actions that have the potential to cause a problem (by throwing a double-fault exception) during exception unwinding if the OperationContext is in an interrupted state.&lt;/p&gt;

&lt;p&gt;The exceptions are the lock acquisitions on  &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c3e78c91c3a86fd6aba44a0b3c97062f55512f56/src/mongo/db/commands/mr.cpp#L1183&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;line 1183&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c3e78c91c3a86fd6aba44a0b3c97062f55512f56/src/mongo/db/commands/mr.cpp#L1503&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;line 1503&lt;/a&gt;. It&apos;s not surprising that the unit tests didn&apos;t trip these lines, because an interrupt has to occur within a narrow window to cause them to execute: during the time that the command gives up its collection lock, either &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c3e78c91c3a86fd6aba44a0b3c97062f55512f56/src/mongo/db/commands/mr.cpp#L1206-L1210&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c3e78c91c3a86fd6aba44a0b3c97062f55512f56/src/mongo/db/commands/mr.cpp#L1550-L1553&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;My understanding is that &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37449&quot; title=&quot;Delete CursorManager&amp;#39;s PlanExecutor registry&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37449&quot;&gt;&lt;del&gt;SERVER-37449&lt;/del&gt;&lt;/a&gt; makes these lock acquisitions unnecessary, and that they are on the chopping block anyway as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37453&quot; title=&quot;Delete PlanStage::dispose()&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37453&quot;&gt;&lt;del&gt;SERVER-37453&lt;/del&gt;&lt;/a&gt;. As part of this work, I&apos;ll delete both ON_BLOCK_EXIT blocks along with the UninterruptibleLockGuards after confirming with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt; about my understanding of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37449&quot; title=&quot;Delete CursorManager&amp;#39;s PlanExecutor registry&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37449&quot;&gt;&lt;del&gt;SERVER-37449&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The dropTempCollections() method also gets called as part of destruction, but the destructor wraps this call in a try block that catches all exceptions, so there is no risk of a double-fault exception. Interrupting a map-reduce that has a temp collection will leave the temp collection in place, which is probably the desired behavior. It will keep interrupt cleanup fast, and the temp collections still get cleaned up later. (The log message says that they get cleaned up when the mongod is restarted; perhaps we should consider a more frequent interval than that to be safe.)&lt;/p&gt;</comment>
                            <comment id="2086432" author="siyuan.zhou@10gen.com" created="Mon, 10 Dec 2018 22:49:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt;, I talked with&#160;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;.&#160;Quoting him on adding UninterruptibleLockGuard in &lt;tt&gt;mr.cpp&lt;/tt&gt;.&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;We decided to put that there after seemingly endless bugs with map reduce getting interrupted. It was way too complicated to bother identifying every single location. Instead we put that there to ensure nothing bad happened until we decided to investigate the failures.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I&apos;ve run three patch builds &lt;a href=&quot;https://evergreen.mongodb.com/version/5c0ec6a3e3c3315e4764a6ea&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;to remove the one on single node&lt;/a&gt;, &lt;a href=&quot;https://evergreen.mongodb.com/build/mongodb_mongo_master_enterprise_rhel_62_64_bit_patch_55e72b015e2aa7297c00db29e4d93451ea61a7ca_5c0ec6c1e3c3315e4764a951_18_12_10_20_04_39&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;to remove the one on shared node&lt;/a&gt;&#160;and &lt;a href=&quot;https://evergreen.mongodb.com/build/mongodb_mongo_master_enterprise_rhel_62_64_bit_patch_55e72b015e2aa7297c00db29e4d93451ea61a7ca_5c0ec711e3c3315e4764ae4b_18_12_10_20_05_50&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;to remove all three including the one in dropTempCollections()&lt;/a&gt;. Surprisingly, I haven&apos;t seen any failure related to map-reduce. This may be because we don&apos;t have a good test coverage on the concurrency of map-reduce, or the recent changes on map-reduce / query have already made it resilient to interrupts. Passing Evergreen doesn&apos;t make me feel comfortable to remove them blindly. I think Query team has more context than Replication team to investigate the impact of the removal and fix the issues when they come out.&lt;/p&gt;</comment>
                            <comment id="2085915" author="david.storch" created="Mon, 10 Dec 2018 18:41:54 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;, my understanding was that the work to investigate why mapReduce&apos;s strong lock acquisitions are not interruptible would fall to the replication team. Did you already complete that investigation?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="553260">SERVER-35365</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 10 Dec 2018 18:41:54 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 1 week, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1294</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 1 week, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>justin.seyster@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huf8av:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu72nz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2672">Query 2019-01-14</customfieldvalue>
    <customfieldvalue id="2751">Query 2019-01-28</customfieldvalue>
    <customfieldvalue id="2752">Query 2019-02-11</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hueuk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>