Core Server
  1. Core Server
  2. SERVER-939

ability to distribute collections in a single db

    Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Critical - P2 Critical - P2
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Planning Bucket A
    • Component/s: Sharding
    • Labels:
      None
    • # Replies:
      93
    • Last comment by Customer:
      true
    • Driver Changes:
      Not Needed

      Description

      The implementation of this feature (without moveCollections) has the following work items:

      • Add new parsing functionality to mongos/mongod for chunk manager logic
      • Build new scatterCollections and createCollection commands
      • Deprecate enableSharding
      • Moderate refactoring of the mongod config reload codebase (d_logic.cpp)

      To implement moveCollections some additional work is probably necessary, but we can leverage much of the cloneCollection framework.

        Issue Links

          Activity

          Hide
          auto (Inactive)
          added a comment -

          Author:

          {u'date': u'2013-01-22T17:33:03Z', u'email': u'shaun.verch@10gen.com', u'name': u'Shaun Verch'}

          Message: SERVER-939 Made source argument of parseBSON const ref
          Branch: master
          https://github.com/mongodb/mongo/commit/24b35b78bc6f7b7e964edfa2ae96813a39aac4f7

          Show
          auto (Inactive)
          added a comment - Author: {u'date': u'2013-01-22T17:33:03Z', u'email': u'shaun.verch@10gen.com', u'name': u'Shaun Verch'} Message: SERVER-939 Made source argument of parseBSON const ref Branch: master https://github.com/mongodb/mongo/commit/24b35b78bc6f7b7e964edfa2ae96813a39aac4f7
          Hide
          auto (Inactive)
          added a comment -

          Author:

          {u'date': u'2013-01-22T18:36:24Z', u'name': u'Shaun Verch', u'email': u'shaun.verch@10gen.com'}

          Message: SERVER-939 Removed unused default parameter from extract function
          Branch: master
          https://github.com/mongodb/mongo/commit/439ce8c8911e33ec47fd1bec9bb6289baf3edfc0

          Show
          auto (Inactive)
          added a comment - Author: {u'date': u'2013-01-22T18:36:24Z', u'name': u'Shaun Verch', u'email': u'shaun.verch@10gen.com'} Message: SERVER-939 Removed unused default parameter from extract function Branch: master https://github.com/mongodb/mongo/commit/439ce8c8911e33ec47fd1bec9bb6289baf3edfc0
          Hide
          auto (Inactive)
          added a comment -

          Author:

          {u'date': u'2013-01-29T16:08:11Z', u'email': u'randolph@10gen.com', u'name': u'Randolph Tan'}

          Message: SERVER-939 abilitiy to distribute collections in a single db

          Port tests from d_chunk_manager_tests.cpp for collection_manager_test.cpp
          Branch: master
          https://github.com/mongodb/mongo/commit/d305bf71dd0c49909c7c59eb3d8c282f9d966128

          Show
          auto (Inactive)
          added a comment - Author: {u'date': u'2013-01-29T16:08:11Z', u'email': u'randolph@10gen.com', u'name': u'Randolph Tan'} Message: SERVER-939 abilitiy to distribute collections in a single db Port tests from d_chunk_manager_tests.cpp for collection_manager_test.cpp Branch: master https://github.com/mongodb/mongo/commit/d305bf71dd0c49909c7c59eb3d8c282f9d966128
          Hide
          JinYan.Zhang
          added a comment - - edited

          Because documents has size limit of 4MB, so we use collections to aggregate records.
          For example:There is a blog cloud(SAAS),
          we create a collection of mongodb dynamiclly for each author, to store their own blog posts privately.
          we may create millions of collections, and we wish to shard on collections level.
          As each author has limited number of posts, so all the data of a collection will shard into one single server.

          We wish to shard on collections dynamically, a use case:
          collections with name ends with 'A' will go to shard1,
          collections with name ends with 'B' will go to shard2,
          etc

          We really expect this feature.
          thks

          Show
          JinYan.Zhang
          added a comment - - edited Because documents has size limit of 4MB, so we use collections to aggregate records. For example:There is a blog cloud(SAAS), we create a collection of mongodb dynamiclly for each author, to store their own blog posts privately. we may create millions of collections, and we wish to shard on collections level. As each author has limited number of posts, so all the data of a collection will shard into one single server. We wish to shard on collections dynamically, a use case: collections with name ends with 'A' will go to shard1, collections with name ends with 'B' will go to shard2, etc We really expect this feature. thks
          Hide
          Asya Kamsky
          added a comment - - edited

          JinYan.Zhang first of all, document size limit is 16MB and not 4MB.

          And you can already distribute collections across shards via tag aware sharding - I have a write-up with some examples here:

          http://askasya.com/post/taggedcollectionbalancing

          In addition you can store all blog posts in a single collection, just identifying it with a specific author and then shard the collection by author.

          Show
          Asya Kamsky
          added a comment - - edited JinYan.Zhang first of all, document size limit is 16MB and not 4MB. And you can already distribute collections across shards via tag aware sharding - I have a write-up with some examples here: http://askasya.com/post/taggedcollectionbalancing In addition you can store all blog posts in a single collection, just identifying it with a specific author and then shard the collection by author.

            People

            • Votes:
              94 Vote for this issue
              Watchers:
              83 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since reply:
                16 weeks, 2 days ago
                Date of 1st Reply: