<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:19:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-8990] Support Replication Between Sharded Clusters</title>
                <link>https://jira.mongodb.org/browse/SERVER-8990</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Please support replication between sharded clusters. This would effectively first require multi-master replication behavior and support to exist.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;One Idea&lt;/b&gt;:&lt;br/&gt;
Good idea to use an existing op-log for cross-cluster replication. Read from op-log from each shard and apply the operation on destination mongos (another datacenter).&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/25055/25055_img1.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/25054/25054_img2.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;The problem is in the same operation will appear in destination cluster op-log and infinite loop will start as a result.&lt;/p&gt;

&lt;p&gt;The solution is to append to document a special marker to specify the appurtenance to the particular cluster. &lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/25053/25053_img3.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;
</description>
                <environment></environment>
        <key id="68547">SERVER-8990</key>
            <summary>Support Replication Between Sharded Clusters</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="backlog-server-repl">Backlog - Replication Team</assignee>
                                    <reporter username="vasmik">Vasily</reporter>
                        <labels>
                            <label>pull-request</label>
                    </labels>
                <created>Fri, 15 Mar 2013 12:42:28 +0000</created>
                <updated>Tue, 6 Dec 2022 05:23:03 +0000</updated>
                            <resolved>Mon, 7 Oct 2019 20:59:09 +0000</resolved>
                                                                    <component>Replication</component>
                    <component>Sharding</component>
                                        <votes>5</votes>
                                    <watches>14</watches>
                                                                                                                <comments>
                            <comment id="2469992" author="steven.vannelli" created="Mon, 7 Oct 2019 20:59:09 +0000"  >&lt;p&gt;Closing this ticket as Won&apos;t Do as the parent Epic is no longer needed at this time.&lt;/p&gt;</comment>
                            <comment id="807136" author="ramon.fernandez" created="Sat, 17 Jan 2015 15:22:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=vasmik&quot; class=&quot;user-hover&quot; rel=&quot;vasmik&quot;&gt;vasmik&lt;/a&gt;, thanks for taking the time to produce a pull request, but unfortunately we cannot merge it at this time: adding a &quot;payload&quot; to every oplog entry may have a huge performance impact and other ill effects on existing deployments, so this is a change that would need very careful consideration and thorough field testing. We&apos;re therefore closing your pull request for now, but keeping the ticket open for future consideration.&lt;/p&gt;

&lt;p&gt;Note that you may insert markers in your oplog at any time, as per &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12119&quot; title=&quot;Add new command to allow applications to append notes to the oplog.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12119&quot;&gt;&lt;del&gt;SERVER-12119&lt;/del&gt;&lt;/a&gt;, provided you use MongoDB version 2.6 or newer.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="291915" author="vasmik" created="Mon, 18 Mar 2013 13:15:33 +0000"  >&lt;p&gt;Hi Scott,&lt;br/&gt;
We understand that we can have a shard key which is DC specific. But if we have data written in one DC we will not be able to modify them in another DC locally. We understand that we can potentially update data from within DC2 directly to DC1. But if DC1 is down these data will not be modifiable. This is the problem which we, and we believe MongoDB in general, should address. Otherwise MongoDB is inferior to rival Cassandra, CouchBase or other NoSQL DBs. Adding DC marker will make op-log DC-aware, allowing other developers to make their own replication.&lt;/p&gt;

&lt;p&gt;About conflict resolution. Other DB providers, e.g. Cassandra, CouchBase etc, solved this problem one way or another. There is industry best practices to resolve conflicts and they can be implemented in replicator app. It could be totally independent from MongoDB. But MongoDB would have a way to make this replication possible if op-log would contain DC marker.&lt;/p&gt;</comment>
                            <comment id="290085" author="vasmik" created="Fri, 15 Mar 2013 14:01:09 +0000"  >&lt;p&gt;Actually we have an external sync-engine that uses this marker. This tool is responsible to conflict-resolution. &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-2956&quot; title=&quot;Master-Master replication&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-2956&quot;&gt;&lt;del&gt;SERVER-2956&lt;/del&gt;&lt;/a&gt; issue exists for almost 2 years without any solution as I can see.&lt;br/&gt;
My pull-request: &lt;a href=&quot;https://github.com/mongodb/mongo/pull/400&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/pull/400&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="290061" author="scotthernandez" created="Fri, 15 Mar 2013 13:30:20 +0000"  >&lt;p&gt;What you are essentially asking for is multi-master replication (see linked issue) plus replication between two sharded clusters. As you have shown, you can build something like this external to our current sharded clusters and replication system (using the same type of architecture), but it is not something which is built-in today.&lt;/p&gt;

&lt;p&gt;Do you ever have the chance in your application to write data which will conflict? What would expect to happen if there was a conflict and the data had to be merged or replaced based on some policy? No need to answer here, but these questions are crux of the multi-master replication challenge and would probably be best discussed in that feature request.&lt;/p&gt;

&lt;p&gt;Currently you can use a single sharded cluster with different shards which have a primary in each data center to make sure that writes (esp. easy with inserts) are always local to the data center; you basically write to the master in your data center. To do this you need to combine a synthetic shard key, if you don&apos;t already have one which isolates writes to the datacenter/region, and shard/chunk tags to keep the data distributed (balanced on) to the data center you wish to write to.&lt;/p&gt;</comment>
                            <comment id="290049" author="vasmik" created="Fri, 15 Mar 2013 13:01:34 +0000"  >&lt;p&gt;&quot;Vasily, this sounds like you want to do replication between two separate sharded clusters across different groups of replica sets in each sharded cluster. Is that the feature you are asking for?&quot;&lt;/p&gt;

&lt;p&gt;Exactly! Because of MongoDB does not support master-master replication by default we can&apos;t share the replica-sets between datacenters (each RS has nodes in both DCs).&lt;br/&gt;
But we wish to get an ability to make writes to both DCs simultaneously. It looks like a bi-directional replication or master-master replication.&lt;br/&gt;
In this case we can set independent clusters for each DC in different configuration.&lt;/p&gt;

&lt;p&gt;Thanks for your attention, Scott!&lt;/p&gt;</comment>
                            <comment id="290036" author="scotthernandez" created="Fri, 15 Mar 2013 12:49:41 +0000"  >&lt;p&gt;Vasily, this sounds like you want to do replication between two separate sharded clusters across different groups of replica sets in each sharded cluster. Is that the feature you are asking for?&lt;/p&gt;

&lt;p&gt;Currently nothing like this is supported by mongodb and the &quot;replication servers&quot; you have listed in your diagrams aren&apos;t a component we have. Can you explain a little more the use-case you are trying to address?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="16167">SERVER-2956</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="25055" name="img1.png" size="45969" author="vasmik" created="Fri, 15 Mar 2013 12:42:28 +0000"/>
                            <attachment id="25054" name="img2.png" size="66237" author="vasmik" created="Fri, 15 Mar 2013 12:42:28 +0000"/>
                            <attachment id="25053" name="img3.png" size="65897" author="vasmik" created="Fri, 15 Mar 2013 12:42:28 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 15 Mar 2013 12:49:41 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 18 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-259</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 18 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-repl</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
            <customfieldvalue>steven.vannelli@mongodb.com</customfieldvalue>
            <customfieldvalue>vasmik</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrn1g7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfysn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6687</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0nxj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>