<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:31:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-13442] mapReduce nonAtomic output option</title>
                <link>https://jira.mongodb.org/browse/SERVER-13442</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;m running some map reduce jobs, specifying the out parameter as: &lt;/p&gt;
{reduce: &amp;lt;outputCollection&amp;gt;, db: &amp;lt;outputDB&amp;gt;}

&lt;p&gt;The output step takes many minutes to complete and seems to lock the entire mongod process while it&apos;s running. I&apos;ve read that the nonAtomic option could perhaps prevent this locking behavior. I&apos;m running multiple map reduce jobs in parallel which all specify the same output collection and db in the reduce step. Does specifying &lt;/p&gt;
{nonAtomic: true}
&lt;p&gt; ensure that the individual documents in the output collection are updated atomically? Could there exist race conditions between the parallel MR reduce operations since they are reducing to the same collection? &lt;/p&gt;</description>
                <environment></environment>
        <key id="128808">SERVER-13442</key>
            <summary>mapReduce nonAtomic output option</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="mathias@mongodb.com">Mathias Stearn</assignee>
                                    <reporter username="gkolpin">Garrett Kolpin</reporter>
                        <labels>
                    </labels>
                <created>Tue, 1 Apr 2014 18:06:40 +0000</created>
                <updated>Wed, 10 Dec 2014 23:07:00 +0000</updated>
                            <resolved>Thu, 24 Jul 2014 22:07:50 +0000</resolved>
                                                                    <component>MapReduce</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="670734" author="ramon.fernandez" created="Thu, 24 Jul 2014 22:07:33 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gkolpin&quot; class=&quot;user-hover&quot; rel=&quot;gkolpin&quot;&gt;gkolpin&lt;/a&gt;, there are no race conditions when running multiple MR jobs and using &lt;tt&gt;reduce&lt;/tt&gt; operations (using &lt;tt&gt;merge&lt;/tt&gt; the behavior is undefined, but looks like you&apos;re not doing this so no need to worry about it).&lt;/p&gt;

&lt;p&gt;As for the global lock, we have &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13552&quot; title=&quot;remove unnecessary global lock during &amp;quot;replace&amp;quot; out action&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13552&quot;&gt;SERVER-13552&lt;/a&gt; to improve the situation, so we&apos;re closing this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13552&quot; title=&quot;remove unnecessary global lock during &amp;quot;replace&amp;quot; out action&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13552&quot;&gt;SERVER-13552&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="670148" author="gkolpin" created="Thu, 24 Jul 2014 16:10:43 +0000"  >&lt;p&gt;Hi Thomas,&lt;/p&gt;

&lt;p&gt;We&apos;re running mongo 2.4.9 with Linux 2.6.32-431.3.1.el6.x86_64 (Centos 6.5).&lt;/p&gt;

&lt;p&gt;It&apos;s still an issue in the sense that we have global locking issues when running our map reduce jobs, however we&apos;ve mitigated the problem by running our map reduce jobs on a separate instance of mongo, which won&apos;t affect our other operations during the time that the map reduce jobs are running. &lt;/p&gt;

&lt;p&gt;Since I opened this issue, I&apos;ve spoken with Andre Spiegel who confirmed that mongo does indeed grab a global lock during map-reduce operations. It was with consultation with him that we decided on the approach we&apos;re now taking. &lt;/p&gt;

&lt;p&gt;It would be nice to have this fixed, but since we&apos;ve been able to work around the locking issue it&apos;s no longer as critical for us as it was.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Garrett&lt;/p&gt;</comment>
                            <comment id="670113" author="thomasr" created="Thu, 24 Jul 2014 15:50:28 +0000"  >&lt;p&gt;Hi Garrett, &lt;/p&gt;

&lt;p&gt;Is this still an issue for you? If so, can you let me know what OS you were running and what version of MongoDB you were using?&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="539221" author="thomasr" created="Mon, 7 Apr 2014 15:38:49 +0000"  >&lt;p&gt;Hi Garrett, &lt;/p&gt;

&lt;p&gt;Further testing indicates that this might mostly be an issue on the Mac OS X platform, and we couldn&apos;t reproduce such a significant performance impact on Linux. Can you confirm what OS you ran your tests on? Can you also let me know what version of MongoDB you were using?&lt;/p&gt;

&lt;p&gt;Regarding your questions about the behavior of multiple concurrent map/reduce jobs reducing to the same collection, I&apos;ll have to follow up on this and will get back to you when I know more. &lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="535765" author="gkolpin" created="Fri, 4 Apr 2014 15:16:38 +0000"  >&lt;p&gt;Thomas, thanks for the reply. Do you have any info regarding how the reduce phase works when multiple concurrent map-reduces are reducing to the same output collection? Is the reduce phase atomic - could data get dropped if one reduce phase writes a document that another one hasn&apos;t seen and overwrites the previous reduce phase&apos;s data?&lt;/p&gt;</comment>
                            <comment id="534283" author="thomasr" created="Thu, 3 Apr 2014 16:25:40 +0000"  >&lt;p&gt;Hi Garrett, &lt;/p&gt;

&lt;p&gt;Thanks for reporting this issue. &lt;/p&gt;

&lt;p&gt;I&apos;ve done some tests and can reproduce the behavior you&apos;re seeing: Running the same map/reduce job multiple times concurrently (output to the same collection) significantly slows the jobs down. In my tests, a single job took 9 seconds and 4 concurrent jobs took 90 seconds to complete.&lt;/p&gt;

&lt;p&gt;The &lt;tt&gt;nonAtomic&lt;/tt&gt; option did not make a difference in my tests. &lt;/p&gt;

&lt;p&gt;Until we have determined the reason for this behavior, I recommend you run only a single job at a time, as this seems to be the fastest option.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="130553">SERVER-13552</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="129454">SERVER-13490</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 3 Apr 2014 15:26:06 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 29 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 29 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>gkolpin</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
            <customfieldvalue>thomas.rueckstiess@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlxmf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrxmtr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>110311</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmqzz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>