<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:19:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-28945] MongoDB load increases when updating 57k large records in one query</title>
                <link>https://jira.mongodb.org/browse/SERVER-28945</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h6&gt;&lt;a name=&quot;OriginalSummary&quot;&gt;&lt;/a&gt;Original Summary&lt;/h6&gt;
&lt;p&gt;MongoDB Crashed while updating 57k records in one query&lt;/p&gt;

&lt;h6&gt;&lt;a name=&quot;OriginalDescription&quot;&gt;&lt;/a&gt;Original Description&lt;/h6&gt;
&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;We have been struggling with the mongoDB server crash in production which is happening due to one update query which is updating 57000 records in one shot. Application is sending this update query 300 times sequentially but mongoDB server is unusually getting 700-800% of CPU Usage which is causing the mongoDB to crash.&lt;/p&gt;

&lt;p&gt;Query which creating the crash to happen:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2017-04-22T18:42:27.896+0000 I WRITE    [conn28248] update hc.Schedule query: { service.bouquetDetails.handle: &quot;dvbBouquet:1.6406E7&quot; } update: { $pull: { service.bouquetDetails: { handle: &quot;dvbBouquet:1.6406E7&quot; } } } keysExamined:57785 docsExamined:57785 nMatched:57785 nModified:57785 keyUpdates:0 writeConflicts:0 numYields:2318 locks:{ Global: { acquireCount: { r: 60104, w: 60104 } }, Database: { acquireCount: { w: 60104 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 118 } }, Collection: { acquireCount: { w: 2319 } }, Metadata: { acquireCount: { w: 57785 } }, oplog: { acquireCount: { w: 57785 } } } 127328ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2017-04-22T18:53:37.395+0000 I WRITE    [conn24466] update hc.Schedule query: { service.bouquetDetails.handle: &quot;dvbBouquet:1.6409E7&quot; } update: { $pull: { service.bouquetDetails: { handle: &quot;dvbBouquet:1.6409E7&quot; } } } keysExamined:57072 docsExamined:57072 nMatched:57072 nModified:57072 keyUpdates:0 writeConflicts:0 numYields:2249 locks:{ Global: { acquireCount: { r: 59322, w: 59322 } }, Database: { acquireCount: { w: 59322 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 100 } }, Collection: { acquireCount: { w: 2250 } }, Metadata: { acquireCount: { w: 57072 } }, oplog: { acquireCount: { w: 57072 } } } 118500ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Cluster Setup:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;We have no sharding.&lt;/li&gt;
	&lt;li&gt;One replica set with three Mongod servers (version: 3.2.8) and two Mongos Server&lt;/li&gt;
	&lt;li&gt;All are running on Linux environment on OpenStack cloud.&lt;/li&gt;
	&lt;li&gt;Application is in java 8 and using mongodb driver 3.0.2&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Questing: Is MongoDB can&apos;t handle 57000 records update in one query or there is a bug in MongoDB 3.2.8 version with about specified query.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Sulabh&lt;/p&gt;</description>
                <environment>MongoDB Server 3.2.8, MongoDB Java Driver 3.0.2, Linux operating system</environment>
        <key id="377104">SERVER-28945</key>
            <summary>MongoDB load increases when updating 57k large records in one query</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="sulabh84">sulabh</reporter>
                        <labels>
                    </labels>
                <created>Mon, 24 Apr 2017 20:27:42 +0000</created>
                <updated>Tue, 25 Apr 2017 15:55:26 +0000</updated>
                            <resolved>Tue, 25 Apr 2017 15:55:26 +0000</resolved>
                                    <version>3.2.8</version>
                                                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="1557355" author="thomas.schubert" created="Tue, 25 Apr 2017 15:55:16 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sulabh84&quot; class=&quot;user-hover&quot; rel=&quot;sulabh84&quot;&gt;sulabh84&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for clarifying that a crash did not occur. I&apos;ve updated the ticket summary accordingly. Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt; or &lt;a href=&quot;http://stackoverflow.com/questions/tagged/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Stack Overflow with the &lt;tt&gt;mongodb&lt;/tt&gt; tag&lt;/a&gt;. A question like this involving more discussion would be best posted on the mongodb-users group.&lt;/p&gt;

&lt;p&gt;See also our &lt;a href=&quot;https://docs.mongodb.org/manual/support&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Technical Support page&lt;/a&gt; for additional support resources.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1556936" author="sulabh84" created="Tue, 25 Apr 2017 08:01:25 +0000"  >&lt;p&gt;Just got the update that there was a high load on mongo db because of the size of the Document which is been updated is high 33K and the number of documents updated are 57000. So that came to high CPU usage 700-800%. &lt;/p&gt;

&lt;p&gt;But mongoDB has not crashed but high load. This is also pointing to the storage type which is used in production.&lt;/p&gt;

&lt;p&gt;But still the 700-800% CPU usage is in question.&lt;/p&gt;</comment>
                            <comment id="1556904" author="thomas.schubert" created="Tue, 25 Apr 2017 05:50:14 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sulabh84&quot; class=&quot;user-hover&quot; rel=&quot;sulabh84&quot;&gt;sulabh84&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;ve created a secure &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/bbbfeff8-d68f-4a83-96f7-348d60d69e46.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upload portal&lt;/a&gt; for the log files. Files uploaded to this portal are only visible to MongoDB employees and are routinely deleted after some time.&lt;/p&gt;

&lt;p&gt;I understand that sharing log files may be a challenge. However, in order for us to better understand what is going on here, and appropriately investigate this issue, we would need to see how this mongod crashed (i.e. the backtrace) and what events preceded the crash.&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Thomas &lt;/p&gt;</comment>
                            <comment id="1556903" author="sulabh84" created="Tue, 25 Apr 2017 05:35:44 +0000"  >&lt;p&gt;Hi Ramon, &lt;/p&gt;

&lt;p&gt;It is hard to share the logs since mongoDB node is running on customer site. But what i can give as information is bouquetDetails is a sub document of Service Document and bouquetDetails is an array. So, to update entry we used to use $pull the record and then we used to have another query with $addToSet to add the new document in the array.&lt;/p&gt;

&lt;p&gt;These query happen 300 time in a sequential way and whenever this process triggered by the application we used to high spike in the mongoDB node of 700-800% of CPU utilisation.&lt;/p&gt;

&lt;p&gt;Is $pull and $addToSet are not recommended to used when the records are higher which match the query criteria?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Sulabh&lt;/p&gt;</comment>
                            <comment id="1556732" author="ramon.fernandez" created="Mon, 24 Apr 2017 20:43:38 +0000"  >&lt;p&gt;Can you please provide the logs for the affected node from the last restart until the crash you describe?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 24 Apr 2017 20:30:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 42 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 42 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
            <customfieldvalue>sulabh84</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht6btr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsyl2n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrircf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>