<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:23:16 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-10474] How does replication work and what is the performance bottlenecks?</title>
                <link>https://jira.mongodb.org/browse/SERVER-10474</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We&apos;ve been having issues with replication lag.&lt;br/&gt;
Had to manually restart one of two slaves and use db.adminCommand(&lt;/p&gt;
{replSetMaintenance:true}
&lt;p&gt;), let it sync up and then do the same for the other slave. Only then were both able to catch up with the oplog.&lt;/p&gt;

&lt;p&gt;Why does it help to restart mongodb for it to start catching up when 1800+ seconds behind? Did not help to set maintenance mode.&lt;/p&gt;

&lt;p&gt;What&apos;s the bottlenecks of replication?&lt;br/&gt;
Does all &quot;repl writer workers&quot; have to wait for the one writer thread to get the oplog replication done?&lt;br/&gt;
Did not seem to utilize much of the resources available when the replication was stalled.&lt;/p&gt;

&lt;p&gt;Will the maximum capacity to replicate one database depend on how much one cpu core can handle? How do we monitor this limitation?&lt;/p&gt;</description>
                <environment>Ubuntu LTS</environment>
        <key id="85273">SERVER-10474</key>
            <summary>How does replication work and what is the performance bottlenecks?</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="stephen.steneker@mongodb.com">Stennie Steneker</assignee>
                                    <reporter username="balboah">Johnny Boy</reporter>
                        <labels>
                            <label>performance</label>
                            <label>replicaset</label>
                    </labels>
                <created>Fri, 9 Aug 2013 08:57:08 +0000</created>
                <updated>Wed, 10 Dec 2014 23:11:54 +0000</updated>
                            <resolved>Mon, 19 Aug 2013 12:16:00 +0000</resolved>
                                    <version>2.2.3</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="406175" author="stennie" created="Mon, 19 Aug 2013 12:16:00 +0000"  >&lt;p&gt;Hi Johnny,&lt;/p&gt;

&lt;p&gt;As we haven&apos;t heard from you in a while I&apos;m going to close this issue.&lt;/p&gt;

&lt;p&gt;If you have further support questions, the community forums like &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-users&lt;/a&gt; and StackOverflow are a better starting point.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Stephen&lt;/p&gt;</comment>
                            <comment id="401646" author="stennie" created="Tue, 13 Aug 2013 04:44:31 +0000"  >&lt;p&gt;Hi Johnny,&lt;/p&gt;

&lt;p&gt;You should not need to set maintenance mode or restart the secondary in order to have replication continue successfully.  By chance are you running any scripts or commands to kill long running operations on the server?&lt;/p&gt;

&lt;p&gt;Can you open a ticket in the &lt;a href=&quot;https://jira.mongodb.org/secure/CreateIssue!default.jspa&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;SUPPORT (Community Private)&lt;/a&gt; project and attach your mongod logs?  Please reference &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-10474&quot; title=&quot;How does replication work and what is the performance bottlenecks?&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-10474&quot;&gt;&lt;del&gt;SERVER-10474&lt;/del&gt;&lt;/a&gt; in the description as well.  Information in the SUPPORT project is private and only shared with the 10gen team.&lt;/p&gt;

&lt;p&gt;As far as more details on replication mechanics, I would suggest reviewing the &lt;a href=&quot;http://docs.mongodb.org/manual/core/replication/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Replication documentation in the manual&lt;/a&gt; as well as:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;http://www.10gen.com/presentations/basic-replication-mongodb-0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Introduction to Replication &amp;amp; Replica Sets&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;http://www.10gen.com/presentations/advanced-replication-interals&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Advanced Replication Internals&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;http://www.kchodorow.com/blog/2010/10/12/replication-internals/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Kristina Chodorow&apos;s blog posts on replication internals&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Regards,&lt;br/&gt;
Stephen&lt;/p&gt;</comment>
                            <comment id="400913" author="balboah" created="Mon, 12 Aug 2013 12:04:25 +0000"  >&lt;p&gt;Hello!&lt;/p&gt;

&lt;p&gt;Thank you for the sum up. I will use the user group next time.&lt;/p&gt;

&lt;p&gt;Yes it is the hosts in the MMS group. The pattern is that when we do deploys / having peak time in traffic where the most amount of writes are being performed the replication lag keeps growing and have a hard time catching up without manual intervention.&lt;br/&gt;
It seemed strange that there was no progress in the amount of seconds behind even after setting maintenance mode. Only after restart did it finally progress.&lt;/p&gt;

&lt;p&gt;Is there anything in particular I should look for in the logs?&lt;br/&gt;
Do you need the queries from the log or can I strip those out? There might be somewhat private data in there, if you need the whole shebang can I ask for a pgp key of yours?&lt;/p&gt;

&lt;p&gt;As for the technicality on how the replication works, is there any detailed documentation I can read about that?&lt;/p&gt;

&lt;p&gt;Thank you!&lt;/p&gt;</comment>
                            <comment id="400902" author="stennie" created="Mon, 12 Aug 2013 11:32:16 +0000"  >&lt;p&gt;Hi Johnny,&lt;/p&gt;

&lt;p&gt;The SERVER project is for reporting bugs or feature suggestions for the MongoDB server.&lt;/p&gt;

&lt;p&gt;For MongoDB-related support discussion you should post on the mongodb-users group (&lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user&lt;/a&gt;) or Stack Overflow.  A question like this involving more discussion would be best posted on the mongodb-users group.&lt;/p&gt;

&lt;p&gt;In regards to replication lag, you would need to be more specific in terms of what you are seeing.  For example, is this sustained replication lag or just apparent short jumps in replication lag.&lt;/p&gt;

&lt;p&gt;Assuming you are referring to hosts in the MMS group linked to your user account, it looks like you have had only one brief bump in replication lag over the past week rather than a sustained problem.  Without seeing the full logs it is hard to know what else was happening at the time, but I would suspect that a resource issue such as networking or I/O contention could have affected your replication.&lt;/p&gt;

&lt;p&gt;If you would like to attach your logs here for review we may be able to provide more insight.  I would note that issues and attachments in the SERVER project are publicly visible though.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Stephen&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 12 Aug 2013 11:32:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 26 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 26 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>balboah</customfieldvalue>
            <customfieldvalue>stephen.steneker@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmk0f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrs4w7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>78018</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hspd7j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>