Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3088

Combine Replica Sets and Shards to a single system, using RAID5 style parity

    • Replication

      As discussed at http://groups.google.com/group/mongodb-user/browse_thread/thread/c18a9d181de2e5f6 , it would be tremendously useful for us to be able to use a RAID-5/6 style distributed parity system to combine ReplicaSets with Shards.

      The current system of sharding and using replica sets requires that we have 2x servers as we have shards.
      While this does gives us is effectively RAID0+1 level protection.
      Every bit is replicated to another server that is ensuring it's safe.

      It would dramatically reduce the number of servers that large deployments need, if we could instead use RAID-5/6 style protection.
      The RAID5 system would store and distribute a parity block, that allows the other blocks to be reconstructed, using the remaining data, and XOR.

      While I'm sure you know how RAID works, it might be helpful to quickly review the exact mechanism-
      http://www.scottklarr.com/topic/23/how-raid-5-really-works/

      So in this system, with 5 shards, we'd use six servers.
      Each shard would store 1/5th of the data, plus a parity block.

      When any one of the systems went down, we could reconstruct it, by using the data remaining on the other 5.

      You can increase redundancy, by increasing the number of parity blocks.

      RAID6 uses the same system, but two parity blocks, to increase the overall reliability of the system, and ensure it can handle two
      failures at the same time.

      If Mongo were to support such a system, companies could deploy dramatically fewer servers, while maintaining a very high level of reliability and failover.

            Assignee:
            backlog-server-repl [DO NOT USE] Backlog - Replication Team
            Reporter:
            savewave Colin Davis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: