<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:58:40 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-1998] Support cluster-wide backup option(s)</title>
                <link>https://jira.mongodb.org/browse/SERVER-1998</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Create some option to do a consistent backup of config + shards at the same time. Preferably this would be initiated from a single point (mongos while no balancing is going on) and would hit specified replica nodes in each shard/config replicaset/MS (if they are RS/MS) to reduce the load on the live system.&lt;/p&gt;

&lt;p&gt;In a multi-node cluster it is important to take a consistent snapshot in time that can be restored.&lt;/p&gt;

&lt;p&gt;Oh, yeah, and there needs to be a restore options as well. &lt;/p&gt;</description>
                <environment></environment>
        <key id="13490">SERVER-1998</key>
            <summary>Support cluster-wide backup option(s)</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="scotthernandez">Scott Hernandez</reporter>
                        <labels>
                    </labels>
                <created>Mon, 25 Oct 2010 06:14:34 +0000</created>
                <updated>Wed, 24 Jun 2015 17:16:14 +0000</updated>
                            <resolved>Wed, 24 Jun 2015 17:16:14 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>4</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="949469" author="ian@10gen.com" created="Wed, 24 Jun 2015 17:16:14 +0000"  >&lt;p&gt;Hi everyone, thanks for your input and apologies for the delay in responding to this feature request.&lt;/p&gt;

&lt;p&gt;I&apos;m closing this as Won&apos;t Fix since it is not a functionality that we intend to build into the database itself, and instead recommend using the &lt;a href=&quot;https://www.mongodb.com/cloud/backup&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Cloud Manager Backup&lt;/a&gt; or &lt;a href=&quot;https://www.mongodb.com/products/ops-manager&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Ops Manager&lt;/a&gt; to maintain cluster-wide backups.&lt;/p&gt;</comment>
                            <comment id="25119" author="plasma" created="Thu, 3 Mar 2011 02:15:23 +0000"  >&lt;p&gt;Was just reading &lt;a href=&quot;http://groups.google.com/group/mongodb-user/browse_thread/thread/3388b378125e9978&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user/browse_thread/thread/3388b378125e9978&lt;/a&gt; and it reminded me of this ticket.&lt;/p&gt;

&lt;p&gt;The problem as I understand it now is that taking a backup of Shard #1 and restoring its backup an hour later (if it were to crash) would not be good enough:&lt;br/&gt;
a) Chunks originally on this shard (and in the backup) may not exist on the shard anymore (balancer moved them to another shard)&lt;br/&gt;
b) Chunks allocated to this shard were never in the previous backup, but were since moved to this shard. Their data is however available in a different shards backup (but that shard no longer owns these chunks)&lt;/p&gt;

&lt;p&gt;Thinking out loud about this; what about a backup system where:&lt;br/&gt;
a) You ask the mongo cluster to perform a backup system wide. Its an operation that happens for the entire cluster.&lt;br/&gt;
b) Each shard writes its local backup to a specific directory (or uploads to S3, whatever) independently in parallel&lt;/p&gt;

&lt;p&gt;Assuming then after this backup is taken; Shard #1 catches on fire and you need to restore Shard #1 after buying new servers:&lt;br/&gt;
a) You ask the mongo cluster to restore Shard #1&lt;br/&gt;
b) Mongod Shard #1 primary (for example) asks the config servers where Shard #1&apos;s backup data should be, based on the latest cluster backup available&lt;br/&gt;
c) Config servers tell Shard #1 primary:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;You need to download /backup/shard001/ data files (majority of your shard&apos;s data is here)&lt;/li&gt;
	&lt;li&gt;You need to also then grab a few chunks from /backup/shard002/ because between the time the last backup was taken, and when Shard #1 caught fire, you had some more chunks allocated to you that aren&apos;t in your original Shard #1 backup (but are available in shard002&apos;s backup files - the original chunk owner)&lt;/li&gt;
	&lt;li&gt;You need to ignore restoring chunks XXX from your backup files because they were since given a new owner on a different shard, and so you don&apos;t need to try restoring them as you don&apos;t own them anymore&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Basically; let the mongo cluster be backup-aware and know how to restore data even if chunks have since moved around.&lt;/p&gt;

&lt;p&gt;You just need to make sure enough backup space is available.&lt;/p&gt;

&lt;p&gt;Thoughts?&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Andrew&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="23580" author="dwight_10gen" created="Thu, 10 Feb 2011 17:34:29 +0000"  >&lt;p&gt;in a system that&apos;s been up a while, having the balancer off even for a full 24 hours might be ok.  with the balancer off backups of the whole cluster would be ok then?&lt;/p&gt;</comment>
                            <comment id="23202" author="plasma" created="Sun, 6 Feb 2011 13:36:45 +0000"  >&lt;p&gt;Perhaps a command to initiate a backup of the cluster/shards would be better?&lt;/p&gt;

&lt;p&gt;A single admin command could be issued to the cluster to perform a backup.&lt;/p&gt;

&lt;p&gt;This would tell each shard master (or perhaps an up to date slave) to perform a local server backup and report where that backup is stored on disk locally back to the requesting client (or update a cluster wide status document) on how its going / when its finished.&lt;/p&gt;

&lt;p&gt;You could then perhaps have the server operator automate uploading the backups offsite or to Amazon S3 via SSH to each server etc.&lt;/p&gt;</comment>
                            <comment id="19590" author="eliot" created="Mon, 25 Oct 2010 06:34:12 +0000"  >&lt;p&gt;Most people use sharding where there data gets large - so I disagree that not being able to backup to a single machine will be common.&lt;/p&gt;

&lt;p&gt;we should doc the correct procedure in the wiki&lt;/p&gt;</comment>
                            <comment id="19589" author="scotthernandez" created="Mon, 25 Oct 2010 06:29:55 +0000"  >&lt;p&gt;Yep, I had thought about all those issues, but I suspect many people will not have that much data. I would guess for every 1 cluster that is too big to backup there will at least 20 that can.&lt;/p&gt;

&lt;p&gt;If nothing else this should be documented on the wiki/docs.&lt;/p&gt;

&lt;p&gt;Maybe the easiest option would be to use a (non-production) mongos instance where the backup runs with a slaveOk option. That would require mongodump to run with slaveOk on the query which sounds like another issue to create. Also, wouldn&apos;t parts of the config db need to skipped? Cause you would want to setup the cluster before restoring the data + indexes.&lt;/p&gt;</comment>
                            <comment id="19587" author="eliot" created="Mon, 25 Oct 2010 06:18:07 +0000"  >&lt;p&gt;Also - mongodump through mongos will have every object even if migrates are happening live.&lt;/p&gt;</comment>
                            <comment id="19586" author="eliot" created="Mon, 25 Oct 2010 06:17:38 +0000"  >&lt;p&gt;The reason we don&apos;t support a general option like this is that in many sharded systems, its actually impossible to do so since the data can&apos;t fit on a single host.&lt;/p&gt;

&lt;p&gt;Currently you can turn off the balancer, and then backup the config and each shard independently.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="25059">SERVER-4306</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 25 Oct 2010 06:17:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 34 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 34 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>plasma</customfieldvalue>
            <customfieldvalue>dwight@mongodb.com</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrpckn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrg0cv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6606</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrljt3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>