<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:25:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-11159] Secondary stuck during index building</title>
                <link>https://jira.mongodb.org/browse/SERVER-11159</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;2.4.1 replica set&lt;br/&gt;
Customer build a index on the primary as background&lt;br/&gt;
The secondary built the index and then got the following error&lt;br/&gt;
DR102 too much data written uncommitted 314.9MB &lt;/p&gt;

&lt;p&gt;After this secondary was at 100% cpu and not reachable by other members.&lt;br/&gt;
I deleted the secondary data and resynced but same issue.&lt;br/&gt;
Then I took the server offline from replica set and deleted the added index. After that the server was fine. The logs are attached from the secondary&lt;/p&gt;</description>
                <environment></environment>
        <key id="93975">SERVER-11159</key>
            <summary>Secondary stuck during index building</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="dharshanr@scalegrid.net">Dharshan Rangegowda</reporter>
                        <labels>
                    </labels>
                <created>Sat, 12 Oct 2013 16:39:54 +0000</created>
                <updated>Wed, 10 Dec 2014 23:19:11 +0000</updated>
                            <resolved>Fri, 14 Feb 2014 15:55:14 +0000</resolved>
                                    <version>2.4.1</version>
                                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="451549" author="ranjay.krishna" created="Tue, 5 Nov 2013 15:21:19 +0000"  >&lt;p&gt;Hi Dharshan,&lt;/p&gt;

&lt;p&gt;bgsync refers to the rsBackgroundSync. &lt;/p&gt;

&lt;p&gt;It appears that the secondary is under heavy load because it is building an index in the foreground, which is a blocking command. When the bgsync is full, it will stall the secondary, causing it to not be responsive to the secondary. &lt;/p&gt;</comment>
                            <comment id="451254" author="dharshanr@scalegrid.net" created="Mon, 4 Nov 2013 22:26:16 +0000"  >&lt;p&gt;Hi Ranjay,&lt;/p&gt;

&lt;p&gt;Thats what I ended up doing. When you say &quot;bgsync&quot; buffer do you mean the oplog? Also if it was not able to sync shouldn&apos;t the server have gone into recovering state? I noticed that the process got stuck with 100% cpu usage and ended up being not responsive to the other servers. The user shouldn&apos;t be able to do this to the server.&lt;/p&gt;</comment>
                            <comment id="451069" author="ranjay.krishna" created="Mon, 4 Nov 2013 19:18:35 +0000"  >&lt;p&gt;When you build an index on a primary with background=true, it also builds the index on secondaries in the foreground. Since it takes a long time to build the index when running on background, there will be a lot of data that the secondary will have to resync. Since the bgsync buffer is capped at 256 MB, and the data you need resynced is 314.9MB, it will cause the OS to fill up its memory and swap space.&lt;/p&gt;

&lt;p&gt;Note that foreground index build is a blocking operation on the secondary.&lt;br/&gt;
Also, DR102 is a warning not an error. It is issued in dur_commitjob.cpp. I believe for journal sync it can go as high as 512MB at which point mongo will abort.&lt;/p&gt;

&lt;p&gt;I would recommend taking down the secondaries one by one and building the index and adding them back to the replica set. and then finally doing the same to the primary. This is described &lt;a href=&quot;http://docs.mongodb.org/manual/core/index-creation/#building-indexes-on-secondaries&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="32531" name="mongod.log" size="103264947" author="dharshanr@scalegrid.net" created="Sat, 12 Oct 2013 16:39:54 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 4 Nov 2013 19:18:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 15 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 15 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dharshanr@scalegrid.net</customfieldvalue>
            <customfieldvalue>ranjay.krishna</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmbyv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrti5z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>86034</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsp8s7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>