<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:43:35 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36597] primary still contact removed member and now stepdown when major members are down</title>
                <link>https://jira.mongodb.org/browse/SERVER-36597</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;all details have been provide in&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36512&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-36512&lt;/a&gt;&#160;but this topic focus on 2 new found&#160;issues&#160;&lt;/p&gt;

&lt;p&gt;I am confused with Nick&apos;s feedback.&#160;If the Nick&apos;s answer is right, then where are the down major members? because of the replset has 1 primary 2 secondaries and 1 arbiter, only one secondary was unreadable when the issue is found.&lt;/p&gt;

&lt;p&gt;I did more investigation on the mongod.log find another 2 issues:&#160;&lt;/p&gt;

&lt;p&gt;&#160; &lt;b&gt;issue&lt;/b&gt;: primary still contact removed 3 members, fixed by restart mongod service&lt;/p&gt;

&lt;p&gt;&#160; &lt;b&gt;issue&lt;/b&gt;: primary did not step down if it still thinks the replset has 6 data bearing members and 1 arbiter and 4 data bearing members are down.&lt;/p&gt;

&lt;p&gt;&#160;&lt;br/&gt;
secondary : 172.31.54.204 (primary when the issue happen on Aug 6)&lt;br/&gt;
arbiter : ip-172-31-5-208 (was 3.4.7 when the issue happen on Aug 6)&lt;br/&gt;
old member 3.4.7 : ip-172-31-12-59 (removed from replset July 23)&lt;br/&gt;
old member 3.4.7 : ip-172-31-20-52 (removed from replset July 23)&lt;br/&gt;
old member 3.4.7 : ip-172-31-46-24 (removed from replset July 23)&lt;br/&gt;
secondary : ip-172-31-66-130 (unreachable when the issue happen on Aug 6)&lt;br/&gt;
primary : ip-172-31-82-157 (secondary when the issue happen on Aug 6)&lt;br/&gt;
secondary : ip-172-31-67-188 (added afer the issue happen)&lt;/p&gt;

&lt;p&gt;All 3.6.6 now&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="586779">SERVER-36597</key>
            <summary>primary still contact removed member and now stepdown when major members are down</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="nick.brewer">Nick Brewer</assignee>
                                    <reporter username="brucezu">Bruce Zu</reporter>
                        <labels>
                    </labels>
                <created>Fri, 10 Aug 2018 20:04:25 +0000</created>
                <updated>Tue, 4 Sep 2018 03:02:38 +0000</updated>
                            <resolved>Mon, 13 Aug 2018 16:21:00 +0000</resolved>
                                    <version>3.6.6</version>
                                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1974429" author="nick.brewer" created="Mon, 13 Aug 2018 18:21:33 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=brucezu&quot; class=&quot;user-hover&quot; rel=&quot;brucezu&quot;&gt;brucezu&lt;/a&gt; The arbiter does not count toward the read concern majority. With one secondary down in the setup you just described, you would not be able to fulfill a majority read concern. This is not a bug.&#160;&lt;/p&gt;

&lt;blockquote&gt;

&lt;p&gt;But In my case, the primary member actively connect the removed member. This is not expected. it should be a bug. After reboot mongod&#160;service. this issue disappears.&#160;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;As I mentioned previously, we have seen this behavior in the past - however it does not mean that the node that the primary is attempting to connect to is still considered to be a member of the replica set. This is unrelated to your issue of not being able to fulfill a read concern majority, and I&apos;ve linked to the current SERVER tickets that we have tracking improvements in this area.&#160;&lt;/p&gt;

&lt;p&gt;-Nick&lt;/p&gt;</comment>
                            <comment id="1974410" author="brucezu" created="Mon, 13 Aug 2018 18:05:09 +0000"  >&lt;p&gt;Hi Nick&lt;/p&gt;

&lt;p&gt;The output of&#160;&lt;tt&gt;rs.status() shows 4 members in the resplet&lt;/tt&gt;&lt;/p&gt;
&lt;h5&gt;&lt;a name=&quot;secondary%3A172.31.54.204%28primarywhentheissuehappenonAug6%29&quot;&gt;&lt;/a&gt;secondary : 172.31.54.204 (primary when the issue happen on Aug 6)&lt;/h5&gt;
&lt;p&gt;arbiter : ip-172-31-5-208 (was 3.4.7 when the issue happen on Aug 6)&lt;br/&gt;
secondary : ip-172-31-66-130 (unreachable when the issue happen on Aug 6)&lt;br/&gt;
primary : ip-172-31-82-157 (secondary when the issue happen on Aug 6)&lt;/p&gt;

&lt;p&gt;This also can be tracked from mongod.log&#160;&lt;/p&gt;

&lt;p&gt;The question here is only one for them, 1/3,&#160; was unreachable.&#160;&lt;/p&gt;

&lt;p&gt;major data bearing members are still available&lt;/p&gt;

&lt;p&gt;but&#160;lookaside&#160;table started to grow.&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;In short, I think there is a bug: &lt;b&gt;lookaside table is triggered to grow when major data bearing&#160;members are still available.&lt;/b&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;by the way, Test show when a member is removed from replset.&#160; the removed member still tries to connect primary, and primary accept the connection&#160;from the removed member. But primary will never connect the removed member.&lt;/p&gt;

&lt;p&gt;But In my case, the primary member actively connect the removed member. This is not expected. it should be a bug. After reboot mongod&#160;service. this issue disappears.&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;


&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="1974218" author="nick.brewer" created="Mon, 13 Aug 2018 16:20:32 +0000"  >&lt;p&gt;&#160;The primary may have continued to attempt to reach out to the removed nodes, but this does not mean that they were still a part of the replica set. The output of &lt;tt&gt;rs.status()&lt;/tt&gt; should confirm this.&#160;The behavior you&apos;re describing has been seen in the past, and there are a number of tickets currently tracking work that is going into improving the replica set node removal process:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36415&quot; title=&quot;Add dropConnections(const HostAndPort&amp;amp;) to the EgressTagCloser&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36415&quot;&gt;&lt;del&gt;SERVER-36415&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36416&quot; title=&quot;Add a dropConnections administrative command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36416&quot;&gt;&lt;del&gt;SERVER-36416&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36417&quot; title=&quot;Drop pooled connections to nodes no longer in the replica set after a reconfig&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36417&quot;&gt;SERVER-36417&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;-Nick&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 11 Aug 2018 11:33:03 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 26 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 26 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>brucezu</customfieldvalue>
            <customfieldvalue>nick.brewer</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu58fz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htvu53:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu4upb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>