<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:03:26 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-23456] high latency on simple mongo queries during map-reduce operations</title>
                <link>https://jira.mongodb.org/browse/SERVER-23456</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Mongo-DB in spite of having adequate resources ( 4 - Core Box ) with single map-reduce operations seems to be chocking down other operations. In our simple tests , with simple collection having  400 records , a select takes more than  16 seconds.  The map-reduce is running entirely on a different collection. Also we noticed node is unable to connect - resets the connection and on a new connection query is run which normally takes fraction of a second now takes nearly 16 seconds to comeback with a count of &amp;lt; 300 records. &lt;/p&gt;

&lt;p&gt;Conn5865 - Command Count takes 16509 ms while map reduce is running. Under normal load conditions &amp;lt; 100 milliseconds. &lt;/p&gt;

&lt;p&gt;, acquireWaitCount: &lt;/p&gt;
{ r: 3 }
&lt;p&gt;, timeAcquiringMicros: &lt;/p&gt;
{ r: 7055 }
&lt;p&gt; }, Collection: { acquireCount: &lt;/p&gt;
{ r: 11 }
&lt;p&gt; } } protocol:op_query 135ms&lt;br/&gt;
2016-03-31T14:58:11.322-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt; CMD: drop dim3.tmp.mr.whois.tech_postalcode_56cc44bbf2e5301119d51e37s_3924&lt;br/&gt;
2016-03-31T14:58:11.371-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 172.31.32.189:37316 #6167 (363 connections now open)&lt;br/&gt;
2016-03-31T14:58:16.007-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R: (3/3) Final Reduce Progress: 50700/54200 93%&lt;br/&gt;
2016-03-31T14:58:19.030-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 8100/54200 14%&lt;br/&gt;
2016-03-31T14:58:22.045-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 16600/54200 30%&lt;br/&gt;
2016-03-31T14:58:25.028-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 24900/54200 45%&lt;br/&gt;
2016-03-31T14:58:26.190-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 172.31.32.189:37317 #6168 (364 connections now open)&lt;br/&gt;
2016-03-31T14:58:26.191-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6166&amp;#93;&lt;/span&gt; end connection 172.31.32.189:37315 (363 connections now open)&lt;br/&gt;
2016-03-31T14:58:28.004-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 32800/54200 60%&lt;br/&gt;
2016-03-31T14:58:31.015-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 41300/54200 76%&lt;br/&gt;
2016-03-31T14:58:34.011-0700 I -        &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt;   M/R Reduce Post Processing Progress: 50000/54200 92%&lt;br/&gt;
2016-03-31T14:58:35.490-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt; CMD: drop dim3.tmp.mr.whois.tech_postalcode_56cc44bbf2e5301119d51e37s_3924&lt;br/&gt;
2016-03-31T14:58:35.490-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6167&amp;#93;&lt;/span&gt; command local.oplog.rs command: getMore { getMore: 116290011101, collection: &quot;oplog.rs&quot;, maxTimeMS: 5000, term: 17, lastKnownCommittedOpTime: &lt;/p&gt;
{ ts: Timestamp 1459461496000|3513, t: 17 }
&lt;p&gt; } cursorid:116290011101 keyUpdates:0 writeConflicts:0 exception: operation exceeded time limit code:50 numYields:0 reslen:74 locks:{ Global: { acquireCount: &lt;/p&gt;
{ r: 2 }
&lt;p&gt;, acquireWaitCount: &lt;/p&gt;
{ r: 1 }
&lt;p&gt;, timeAcquiringMicros: &lt;/p&gt;
{ r: 19300914 }
&lt;p&gt; }, Database: { acquireCount: &lt;/p&gt;
{ r: 1 }
&lt;p&gt; }, oplog: { acquireCount: &lt;/p&gt;
{ r: 1 }
&lt;p&gt; } } protocol:op_command 19301ms&lt;br/&gt;
2016-03-31T14:58:35.490-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6167&amp;#93;&lt;/span&gt; end connection 172.31.32.189:37316 (362 connections now open)&lt;br/&gt;
2016-03-31T14:58:35.492-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6168&amp;#93;&lt;/span&gt; end connection 172.31.32.189:37317 (361 connections now open)&lt;br/&gt;
2016-03-31T14:58:35.493-0700 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 172.31.32.189:37318 #6169 (362 connections now open)&lt;br/&gt;
2016-03-31T14:58:35.496-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn5865&amp;#93;&lt;/span&gt; command dim3.scans command: count { count: &quot;scans&quot;, query: { $and: [ &lt;/p&gt;
{ scan_status: &quot;Filter Domains&quot; }
&lt;p&gt; ] } } planSummary: COUNT_SCAN &lt;/p&gt;
{ scan_status: 1.0 }
&lt;p&gt; keyUpdates:0 writeConflicts:0 numYields:1 reslen:62 locks:{ Global: { acquireCount: &lt;/p&gt;
{ r: 4 }
&lt;p&gt;, acquireWaitCount: &lt;/p&gt;
{ r: 1 }
&lt;p&gt;, timeAcquiringMicros: &lt;/p&gt;
{ r: 16503711 }
&lt;p&gt; }, Database: { acquireCount: &lt;/p&gt;
{ r: 2 }
&lt;p&gt;, acquireWaitCount: &lt;/p&gt;
{ r: 1 }
&lt;p&gt;, timeAcquiringMicros: &lt;/p&gt;
{ r: 5971 }
&lt;p&gt; }, Collection: { acquireCount: &lt;/p&gt;
{ r: 2 }
&lt;p&gt; } } protocol:op_query 16509ms&lt;br/&gt;
2016-03-31T14:58:35.499-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt; CMD: drop dim3.tmp.mr.whois.tech_postalcode_56cc44bbf2e5301119d51e37s_3924&lt;br/&gt;
2016-03-31T14:58:35.530-0700 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6097&amp;#93;&lt;/span&gt; command dim3.domainstats command: mapReduce { mapreduce: &quot;whois.tech_postalcode_56cc44bbf2e5301119d51e37s&quot;, map: &quot;function () {&lt;/p&gt;

&lt;p&gt;    var key = this._id;&lt;br/&gt;
    var value=&lt;/p&gt;
{&apos;colors&apos;:this.colors}
&lt;p&gt;;&lt;br/&gt;
    if(value.colors){&lt;br/&gt;
     //coming from aggregate collections&lt;br/&gt;
        va...&quot;, reduce: &quot;function (key, values) {&lt;/p&gt;

&lt;p&gt;    //printjson(values)&lt;br/&gt;
    var reduce ={colors:{whois:{}}};&lt;br/&gt;
    values.forEach(function(value){&lt;br/&gt;
        var properties = Ob...&quot;, verbose: true, out: &lt;/p&gt;
{ reduce: &quot;domainstats&quot; }
&lt;p&gt;, jsMode: true, sort: &lt;/p&gt;
{ _id: 1 }
&lt;p&gt;, readPreference: &quot;primary&quot; } planSummary: COUNT keyUpdates:0 writeConflicts:0 numYields:850 reslen:218 locks:{ Global: { acquireCount: &lt;/p&gt;
{ r: 382226, w: 216813, W: 54201 }
&lt;p&gt; }, Database: { acquireCount: &lt;/p&gt;
{ r: 54635, w: 216807, R: 971, W: 8 }
&lt;p&gt;, acquireWaitCount: &lt;/p&gt;
{ W: 2 }
&lt;p&gt;, timeAcquiringMicros: &lt;/p&gt;
{ W: 24771 }
&lt;p&gt; }, Collection: { acquireCount: &lt;/p&gt;
{ r: 54635, w: 108406 }
&lt;p&gt; }, Metadata: { acquireCount: &lt;/p&gt;
{ w: 108403 }
&lt;p&gt; }, oplog: { acquireCount: &lt;/p&gt;
{ w: 108403 }
&lt;p&gt; } } protocol:op_query 24225ms&lt;/p&gt;</description>
                <environment></environment>
        <key id="276577">SERVER-23456</key>
            <summary>high latency on simple mongo queries during map-reduce operations</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="ppeddada">pavan</reporter>
                        <labels>
                    </labels>
                <created>Thu, 31 Mar 2016 22:19:29 +0000</created>
                <updated>Fri, 1 Apr 2016 09:10:08 +0000</updated>
                            <resolved>Fri, 1 Apr 2016 09:10:08 +0000</resolved>
                                                                    <component>MapReduce</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1222282" author="ramon.fernandez" created="Fri, 1 Apr 2016 09:10:08 +0000"  >&lt;p&gt;Closing this issue as per your latest comment &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ppeddada&quot; class=&quot;user-hover&quot; rel=&quot;ppeddada&quot;&gt;ppeddada&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1221933" author="ppeddada" created="Thu, 31 Mar 2016 23:07:23 +0000"  >&lt;p&gt;Looks like Map-Reduce by default locks the entire database as opposed to locking just the collections that its working on, using the &apos;nonAtomic&apos; option seems to be addressing the issue reported here.  This is not an issue anymore.&lt;/p&gt;</comment>
                            <comment id="1221902" author="ppeddada" created="Thu, 31 Mar 2016 22:24:45 +0000"  >&lt;p&gt;MongoTop output, clearly shows the only two operations running are map-reduce and the query.&lt;/p&gt;

&lt;p&gt;                                                              ns      total       read      write    2016-03-31T14:58:36-07:00&lt;br/&gt;
                                                  local.oplog.rs    68924ms    47146ms    21778ms                             &lt;br/&gt;
                                                      dim3.scans    16609ms    16609ms        0ms                             &lt;br/&gt;
dim3.tmp.mr.whois.tech_postalcode_56cc44bbf2e5301119d51e37s_3924    14638ms    16311ms    -1672ms                             &lt;br/&gt;
                                                dim3.domainstats      129ms        0ms      129ms                             &lt;br/&gt;
                                              admin.system.roles        0ms        0ms        0ms                             &lt;br/&gt;
                                            admin.system.version        0ms        0ms        0ms                             &lt;br/&gt;
                                                    config.locks        0ms        0ms        0ms                             &lt;br/&gt;
                                                 dim3.Loading...        0ms        0ms        0ms                             &lt;br/&gt;
                                                   dim3.accounts        0ms        0ms        0ms                             &lt;br/&gt;
                                                   dim3.activity        0ms        0ms        0ms       &lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 1 Apr 2016 09:10:08 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 45 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 45 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>ppeddada</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkbw7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsjz27:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs9xqv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>