<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:20:21 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-29265] STARTUP2 and WriteConcern=majority don&apos;t work well together</title>
                <link>https://jira.mongodb.org/browse/SERVER-29265</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hi,&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;Problem%3A&quot;&gt;&lt;/a&gt;Problem:&lt;/h2&gt;
&lt;p&gt;We just did a rolling upgrade from 3.0.14 to 3.4.3 and we noticed some strange behavior while doing the initial sync. All writes take 5 seconds to complete, making the app very slow and leading to timeouts and finally exhausted connection pools.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;Oursetup%3A&quot;&gt;&lt;/a&gt;Our set-up:&lt;/h2&gt;
&lt;p&gt;The app is a Play-Scala application using the ReactiveMongo driver (0.12.2). We use w: majority and wtimeout: 5000 as our write concern.&lt;br/&gt;
The DB is a replica set of three data bearing nodes, with 2 arbiters.&lt;br/&gt;
Everything is hosted on AWS EC2, using EBS volumes.&lt;/p&gt;

&lt;p&gt;It&apos;s a pretty big database, the initial sync takes around 1,5 hours, incl. index creation.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;Steps&quot;&gt;&lt;/a&gt;Steps&lt;/h2&gt;
&lt;ul&gt;
	&lt;li&gt;We add the new instances to the replica set (on top of the three other nodes), and they start to do an initial sync (from scratch, so not from an AWS snapshot).&lt;/li&gt;
	&lt;li&gt;Once the new node is in the cluster, all writes start to take exactly 5 seconds (the wtimeout value). We don&apos;t see much load on the app or on the primary though.&lt;/li&gt;
	&lt;li&gt;After digging in to the problem, we reconfigured the new nodes with priority:0 and votes:0 and from that moment onwards the whole app is working properly again.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;So we have a kind of workaround, but having to use this workaround sounds undesirable. It also looks a bit weird to me, since the rest of the cluster is quick, there is no need to wait for the new instances the get a majority at all, right? 3 nodes out of 5 respond quick, so that should be enough to return.&lt;/p&gt;

&lt;p&gt;Also, during the initial sync, I don&apos;t see the need for writes to be sent to the node, since it&apos;s not caught up by far and afterwards it will replay these writes anyway.&lt;/p&gt;

&lt;p&gt;Is it possible to ignore nodes that are in STARTUP2 when replicating writes? Or maybe another solution might be to quickly return a special acknowledgement when the node is in STARTUP2, so the primary can return the write back to the application?&lt;/p&gt;</description>
                <environment></environment>
        <key id="384636">SERVER-29265</key>
            <summary>STARTUP2 and WriteConcern=majority don&apos;t work well together</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="schwerin@mongodb.com">Andy Schwerin</assignee>
                                    <reporter username="jankeesvanandel@gmail.com">Jan-Kees van Andel</reporter>
                        <labels>
                    </labels>
                <created>Thu, 18 May 2017 08:07:39 +0000</created>
                <updated>Fri, 27 Oct 2023 13:54:27 +0000</updated>
                            <resolved>Thu, 18 May 2017 13:38:06 +0000</resolved>
                                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1574354" author="schwerin" created="Thu, 18 May 2017 13:32:29 +0000"  >&lt;p&gt;To prevent w:majority writes from rolling back, it is mandatory that the majority of voting nodes confirm writes before the client responds. Other behaviors could lead to loss of w:majority-confirmed writes in the event of an election. As such, the two-phase process for adding new nodes is advisable, and implemented in MongoDB&apos;s opsmanager and cloudmanager products.&lt;/p&gt;

&lt;p&gt;Having two arbiters is a bit of a surprise. Because you have two arbiters and 3 data-bearing nodes to begin with, all three data bearing nodes must confirm every w:majority write (as opposed to only two of them if you had no arbiters). When you add the 4th data bearing node, the total number of nodes in the system is now 6, and the number of nodes that must confirm majority writes increases from 3 to 4, giving you the behavior you&apos;re experiencing.&lt;/p&gt;

&lt;p&gt;Your current architecture will make it impossible to confirm w:majority writes when 1 data bearing node fails. Given that you have three (soon to be 4) data bearing nodes, you should consider whether the arbiters are providing you with any benefit.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 18 May 2017 13:32:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 38 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 38 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>jankeesvanandel@gmail.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht7m5z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hszqyf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht47vz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>