<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:02:49 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3346] MAX SLAVE LAG - Features to provide a more stable Replication Set under high load</title>
                <link>https://jira.mongodb.org/browse/SERVER-3346</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This problem exists in all current Master-Slave servers that I use today.  MongoDB has the exact same issues with Replication that I see in MySQL.&lt;/p&gt;

&lt;p&gt;The situation is:&lt;/p&gt;

&lt;p&gt;1)  Primay is adding/updating data at a rate that is near or at the MAX for the hardware.&lt;br/&gt;
2)  Seconday server is starting to LAG due to loads other than Replication. ( other Secondary servers may be OK at the same time. ) &lt;br/&gt;
3)  Requests for &quot;read&quot; are allowed to provide more read performance.&lt;/p&gt;

&lt;p&gt;Effect:&lt;/p&gt;

&lt;p&gt;The Seconday starts to LAG and does not keep up with the flow of updates and inserts from the Primary.  LAG just grows and grows.&lt;/p&gt;

&lt;p&gt;Solution:&lt;/p&gt;

&lt;p&gt;Provide ways for the drivers and applications to &quot;back off&quot; and reduce the impact on the Seconday so it can &quot;catch-up&quot;.  For example select a Seconday that has lower LAG.&lt;/p&gt;

&lt;p&gt;Ideas:&lt;/p&gt;

&lt;p&gt;1) Add to the slaveOk request a condition of how long the LAG is allowed to be before the request must be performed by some &quot;other&quot; server.&lt;br/&gt;
2) Do not allow requests from servers that are over MAX_SLAVE_LAG set by each servers conf file.&lt;br/&gt;
3) Push back on the Primary if a high load on ALL Seconday servers would stop replication for all known servers.  Could be a Read-Only mode.&lt;br/&gt;
4) Stop allowing read requests from servers with high Loads or LAGS.&lt;/p&gt;

&lt;p&gt;I know that the Root Cause of this issue is an overloaded cluster.  What I am asking for is a nice easy &quot;push-back&quot; from the MongoDB and not a crash.&lt;/p&gt;

&lt;p&gt;Today a MySQL Slave will also just stop replication if the OpLog runs too long/late.  If the OpLog is very huge this situation can LAG for hours/days.&lt;/p&gt;

&lt;p&gt;If the application is OK with a LAG that is Huge,  this needs to be allowed.  But for applicaitons that require a LAG of say &quot;under 60 sec.&quot; a MongoDB feature that helps provide that service would be a great feature.&lt;/p&gt;

&lt;p&gt;The cause of the LAG may also be a Secondary server that is used for backups and was not working for a moment due to a backup request.  Today all of the Seconday servers get an equal number of requests.&lt;/p&gt;

&lt;p&gt;Please call any time: &lt;br/&gt;
Cell: &#160; &#160;  916-202-1600&lt;br/&gt;
Skype:&#160; EdwardMGoldberg&lt;/p&gt;

&lt;p&gt;Edward M. Goldberg&lt;br/&gt;
&lt;a href=&quot;http://myCloudWatcher.com/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://myCloudWatcher.com/&lt;/a&gt;&lt;br/&gt;
e.m.g. &lt;/p&gt;</description>
                <environment>A collection of servers PRIMARY and SECONDAY servers, where the query load is high on the SECONDAY servers and replication load is also very high.</environment>
        <key id="18810">SERVER-3346</key>
            <summary>MAX SLAVE LAG - Features to provide a more stable Replication Set under high load</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="jesse@mongodb.com">A. Jesse Jiryu Davis</assignee>
                                    <reporter username="mycloudwatcher">Edward M. Goldberg</reporter>
                        <labels>
                            <label>LAG</label>
                            <label>replication</label>
                            <label>slaveOk</label>
                    </labels>
                <created>Wed, 29 Jun 2011 00:16:29 +0000</created>
                <updated>Sat, 23 Jul 2016 12:55:50 +0000</updated>
                            <resolved>Mon, 11 Jul 2016 14:05:28 +0000</resolved>
                                    <version>1.8.1</version>
                                                    <component>Replication</component>
                                        <votes>17</votes>
                                    <watches>26</watches>
                                                                                                                <comments>
                            <comment id="1333409" author="ramon.fernandez" created="Sat, 23 Jul 2016 12:55:50 +0000"  >&lt;p&gt;I have marked this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4936&quot; title=&quot;Server support for &amp;quot;maxStalenessMS&amp;quot; read preference option&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4936&quot;&gt;&lt;del&gt;SERVER-4936&lt;/del&gt;&lt;/a&gt;, since &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12861&quot; title=&quot;Introduce a maxStalenessMS option when querying secondaries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12861&quot;&gt;&lt;del&gt;SERVER-12861&lt;/del&gt;&lt;/a&gt; and this ticket were marked as a duplicate of each other.&lt;/p&gt;

&lt;p&gt;Users interested in this feature can tune to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4936&quot; title=&quot;Server support for &amp;quot;maxStalenessMS&amp;quot; read preference option&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4936&quot;&gt;&lt;del&gt;SERVER-4936&lt;/del&gt;&lt;/a&gt; for updates.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1319892" author="schwerin" created="Mon, 11 Jul 2016 14:05:28 +0000"  >&lt;p&gt;I believe that this request is effectively duplicated by a combination of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12861&quot; title=&quot;Introduce a maxStalenessMS option when querying secondaries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12861&quot;&gt;&lt;del&gt;SERVER-12861&lt;/del&gt;&lt;/a&gt; (for maximum tolerable lag in queries) and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-24980&quot; title=&quot;Secondary back pressure in replication&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-24980&quot;&gt;&lt;del&gt;SERVER-24980&lt;/del&gt;&lt;/a&gt; (for having the primary slow down or restrict writes when secondaries cannot keep up).&lt;/p&gt;</comment>
                            <comment id="1189015" author="kevin.rice@searshc.com" created="Tue, 1 Mar 2016 13:54:39 +0000"  >&lt;p&gt;This also applies to situations where the slave is running on slower hardware and batches are infrequent. &lt;br/&gt;
1.  primary gets big set of updates in a batch for a short time.  Primary is on fast hardware (SSD&apos;s) and can process very fast.&lt;br/&gt;
2.  slave is on spinning disk.  Slave sees oplog and starts to update, takes much longer.&lt;/p&gt;

&lt;p&gt;This is okay if the rate of updates is not big and we are only using the slave as a &quot;backup&quot; server, or a read-assist server during the day, and the updates happen at night.&lt;/p&gt;

&lt;p&gt;BUT:  if the rate of updates gets too large, the slaves can&apos;t catch up by the start of business in the morning, or, perhaps ever.&lt;/p&gt;

&lt;p&gt;Having the mix of hardware would allow cheaper slave servers with the tradeoff that the info might be up to time-lag seconds old.  Once the time-lag was too large, though, there should be a throttle on updates so it all can catch up instead of falling over and dying.&lt;/p&gt;


</comment>
                            <comment id="1119131" author="boylook" created="Mon, 21 Dec 2015 03:21:26 +0000"  >&lt;p&gt;+1&lt;/p&gt;</comment>
                            <comment id="1116937" author="zygimantas@soundest.lt" created="Thu, 17 Dec 2015 11:30:59 +0000"  >&lt;p&gt;+1&lt;/p&gt;</comment>
                            <comment id="421531" author="garo" created="Tue, 10 Sep 2013 07:04:24 +0000"  >&lt;p&gt;A slave can be lagging behind for example when it has been resurrected from backup. Currently DBA needs to alter the configuration and mark the slave to be hidden until it has caught up, a task which this would make obsolete. &lt;/p&gt;

&lt;p&gt;+1&lt;/p&gt;</comment>
                            <comment id="117252" author="colinhowe" created="Tue, 8 May 2012 14:25:54 +0000"  >&lt;p&gt;This would be awesome. We are hosted on EC2 and sometimes are slaves get a little bit more behind than normal due to the hardware playing up. If that happens we want to stop the reads automatically. Currently, we&apos;re thinking of a watch process that changes the slave to &apos;hidden&apos; if the lag goes over a certain value..&lt;/p&gt;</comment>
                            <comment id="52391" author="chrisribe" created="Sat, 3 Sep 2011 21:11:12 +0000"  >&lt;p&gt;Indeed, managing replication lag needs to be added.&lt;br/&gt;
There is no easy way to handle lag currently.&lt;br/&gt;
+1&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="30426">SERVER-4936</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="113994">SERVER-12861</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="30423">SERVER-4935</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 3 Sep 2011 21:11:12 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 29 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 29 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>jesse@mongodb.com</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>chrisribe</customfieldvalue>
            <customfieldvalue>colinhowe</customfieldvalue>
            <customfieldvalue>mycloudwatcher</customfieldvalue>
            <customfieldvalue>boylook</customfieldvalue>
            <customfieldvalue>garo</customfieldvalue>
            <customfieldvalue>kevin.rice@searshc.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
            <customfieldvalue>zygimantas@soundest.lt</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrowrr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfyhr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6114</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0ql3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>