<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:02:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3088] Combine Replica Sets and Shards to a single system, using RAID5 style parity</title>
                <link>https://jira.mongodb.org/browse/SERVER-3088</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;As discussed at &lt;a href=&quot;http://groups.google.com/group/mongodb-user/browse_thread/thread/c18a9d181de2e5f6&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user/browse_thread/thread/c18a9d181de2e5f6&lt;/a&gt; , it would be tremendously useful for us to be able to use a RAID-5/6 style distributed parity system to combine ReplicaSets with Shards.  &lt;/p&gt;

&lt;p&gt;The current system of sharding and using replica sets requires that we have 2x servers as we have shards.&lt;br/&gt;
While this does gives us is effectively RAID0+1 level protection.&lt;br/&gt;
Every bit is replicated to another server that is ensuring it&apos;s safe.&lt;/p&gt;

&lt;p&gt;It would dramatically reduce the number of servers that large deployments need, if we could instead use RAID-5/6 style protection.&lt;br/&gt;
The RAID5 system would store and distribute a parity block, that allows the other blocks to be reconstructed, using the remaining data, and XOR. &lt;/p&gt;

&lt;p&gt;While I&apos;m sure you know how RAID works, it might be helpful to quickly  review the exact mechanism- &lt;br/&gt;
&lt;a href=&quot;http://www.scottklarr.com/topic/23/how-raid-5-really-works/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.scottklarr.com/topic/23/how-raid-5-really-works/&lt;/a&gt; &lt;/p&gt;

&lt;p&gt;So in this system, with 5 shards, we&apos;d use six servers. &lt;br/&gt;
Each shard would store 1/5th of the data, plus a parity block. &lt;/p&gt;

&lt;p&gt;When any one of the systems went down, we could reconstruct it, by using the data remaining on the other 5. &lt;/p&gt;

&lt;p&gt;You can increase redundancy, by increasing the number of parity blocks.&lt;/p&gt;

&lt;p&gt;RAID6 uses the same system, but two parity blocks, to increase the overall reliability of the system, and ensure it can handle two &lt;br/&gt;
failures at the same time. &lt;/p&gt;



&lt;p&gt;If Mongo were to support such a system, companies could deploy  dramatically fewer servers, while maintaining a very high level of  reliability and failover. &lt;/p&gt;</description>
                <environment></environment>
        <key id="16981">SERVER-3088</key>
            <summary>Combine Replica Sets and Shards to a single system, using RAID5 style parity</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="1" iconUrl="https://jira.mongodb.org/images/icons/statuses/open.png" description="">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-repl">Backlog - Replication Team</assignee>
                                    <reporter username="savewave">Colin Davis</reporter>
                        <labels>
                    </labels>
                <created>Thu, 12 May 2011 18:57:49 +0000</created>
                <updated>Tue, 6 Dec 2022 05:43:49 +0000</updated>
                                                            <fixVersion>features we&amp;#39;re not sure of</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="32707" author="savewave" created="Sat, 14 May 2011 03:14:43 +0000"  >&lt;p&gt;I apologize for not understanding-&lt;/p&gt;

&lt;p&gt;Do you mean each Physical Hardware server would have one shard server, and two replica sets for shard servers on other machines? &lt;/p&gt;

&lt;p&gt;From my understanding of Mongo Sharding and replication, as described on the web and at &lt;a href=&quot;http://bit.ly/dm6G9v&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://bit.ly/dm6G9v&lt;/a&gt;&lt;br/&gt;
Mongo replication requires that each shard consists of 2+ replica set members; The primary, and at least one secondary.&lt;/p&gt;

&lt;p&gt;If you had 5 Physical Hardware servers, you could run multiple Mongod instances on each server, giving you redundancy, but it would still require the same number of processes running in total, just spread across fewer machines, right?&lt;br/&gt;
That would increase the RAM requirements on each server, versus using explicit parity that is distributed..&lt;/p&gt;

&lt;p&gt;Do continue our RAID analogy, If I&apos;m understanding you correctly, you seem to be suggesting partitioning the my drives, and then using RAID 0+1 to mirror partitions.&lt;/p&gt;

&lt;p&gt;I still need 2x-1 the server resources that I would with a RAID-5 style setup, no?&lt;/p&gt;

&lt;p&gt;I apologize again if I&apos;m missing something.&lt;br/&gt;
-CPD&lt;/p&gt;</comment>
                            <comment id="32704" author="eliot" created="Sat, 14 May 2011 02:56:21 +0000"  >&lt;p&gt;You can do this manually now.&lt;br/&gt;
So if you have 5 servers, you could have 5 shards.&lt;br/&gt;
Each node will handle 1 master and 2 secondaries.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 14 May 2011 02:56:21 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 40 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 40 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-repl</customfieldvalue>
            <customfieldvalue>savewave</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrozwn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfyr3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6550</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlmcn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>