<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:52:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-19796] Cannot stop and backup hidden secondary without risk of inconsistent data.</title>
                <link>https://jira.mongodb.org/browse/SERVER-19796</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Our offsite backup process operates by stopping the MongoD processes on our Hidden Secondary hosts, then capturing their data by cloning their database folder and copying it offsite. &lt;/p&gt;

&lt;p&gt;The restore process works by stopping the MongoD process, removing the existing data within the database folder, then cloning the offsite data back into the database folder.&lt;/p&gt;

&lt;p&gt;On several occasions now we have had the restored MongoD process never get out of &apos;startup2&apos; as it believes there is data still to be replicated.&lt;/p&gt;

&lt;p&gt;We have followed the documented recommendations for stopping a replica host by issuing &quot;db.adminCommand(&lt;/p&gt;
{shutdown : 1}
&lt;p&gt;)&quot; to the MongoD process instead of stopping the process through upstart.&lt;/p&gt;

&lt;p&gt;The documentation for the shutdown command states that it will not run unless &quot;a&quot; secondary has caught up with the primary. If the wording matches the logic, I suspect that the shutdown command proceeds with stopping the Hidden Secondary MongoD host, even when it is behind, because &quot;A&quot; Secondary is up to date (i.e. the non-hidden secondary witihin the replicaset).&lt;/p&gt;</description>
                <environment>Ubuntu 12.04</environment>
        <key id="224419">SERVER-19796</key>
            <summary>Cannot stop and backup hidden secondary without risk of inconsistent data.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="ramon.fernandez@mongodb.com">Ramon Fernandez Marina</assignee>
                                    <reporter username="dave.muysson@360pi.com">Dave Muysson</reporter>
                        <labels>
                    </labels>
                <created>Thu, 6 Aug 2015 14:44:51 +0000</created>
                <updated>Thu, 6 Aug 2015 19:29:51 +0000</updated>
                            <resolved>Thu, 6 Aug 2015 19:27:07 +0000</resolved>
                                    <version>2.4.14</version>
                                                    <component>Admin</component>
                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="998669" author="ramon.fernandez" created="Thu, 6 Aug 2015 19:26:58 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dave.muysson%40360pi.com&quot; class=&quot;user-hover&quot; rel=&quot;dave.muysson@360pi.com&quot;&gt;dave.muysson@360pi.com&lt;/a&gt;, the documentation states the following about the &lt;a href=&quot;http://docs.mongodb.org/v2.4/reference/command/shutdown/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;shutdown command&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;If the node you&#8217;re trying to shut down is a replica set primary, then the command will succeed only if there exists a secondary node whose oplog data is within 10 seconds of the primary.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;When the command is run on a secondary there&apos;s no wait period until that secondary is caught up. To follow the backup procedure described above you may need to wait until the hidden secondary is caught up. There are also &lt;a href=&quot;http://docs.mongodb.org/manual/core/backups/#mongodb-backup-methods&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;other backup procedures&lt;/a&gt; you may want to consider in case they better fit your needs.&lt;/p&gt;

&lt;p&gt;Please note that the SERVER project is for reporting bugs or feature suggestions for the MongoDB server, and unfortunately we&apos;re not able to provide support here. For MongoDB-related support discussion please post on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt; or &lt;a href=&quot;http://stackoverflow.com/questions/tagged/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Stack Overflow with the &lt;tt&gt;mongodb&lt;/tt&gt; tag&lt;/a&gt;, where your question will reach a larger audience. A question like this involving more discussion would be best posted on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt;. See also our &lt;a href=&quot;https://www.mongodb.org/about/support/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Technical Support page&lt;/a&gt; for additional support resources.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 6 Aug 2015 17:38:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 27 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 27 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dave.muysson@360pi.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkybz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hscd6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Using a replicated MongoD environment with one Primary, on Secondary, and one Hidden Secondary:&lt;br/&gt;
(backup process)&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Issue &quot;db.adminCommand(
{shutdown : 1}
&lt;p&gt;)&quot; to the Hidden Secondary&lt;/p&gt;&lt;/li&gt;
	&lt;li&gt;Clone the data under the database folder to an alternate location&lt;/li&gt;
	&lt;li&gt;Start the Hidden Secondary and allow it to catch up.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;(restore process)&lt;br/&gt;
1. Stop all MongoD hosts under the replicaset&lt;br/&gt;
2. Remove the database data on all three MongoD hosts&lt;br/&gt;
3. On the Hidden Secondary, restore the backup data to its original location&lt;br/&gt;
4. Start the Hidden Secondary MongoD process&lt;br/&gt;
5. Force a new config via rs.reconfig() which only includes the hidden secondary&lt;br/&gt;
6. Add the original Primary and Secondary to the replicaset to replicate back&lt;br/&gt;
7. Reconfigure the replicaset back to Primary, Secondary, Hidden Secondary.&lt;/p&gt;

&lt;p&gt;This will work 95% of the time. The other 5%, you cannot get past step 4 of the restore because the data that was backed up was still behind. The shutdown command did not wait for the hidden secondary to be caught up to the primary before it shut down. &lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlggn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>