Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4000

command to change shard key of a collection

    Details

      Description

      Changing shard keys is fundamentally very expensive, but a helper to do this would be useful. The main thing needed would be to do the operation with good parallelism.

      first cut might require the source collection be read only during the operation.

      might do something like

      • measure what the new distribution would be like by looking at a sampled set of records from the originating collection
      • presplit based on statistics above
      • cluster wide copy of data from src to dest collection
      • build the index(es) for dest after the copy to make things as fast as possible

      i suppose this is just a better version of cloneCollection which we'll want anyway.

        Issue Links

          Activity

          Hide
          tubededentifrice Vincent added a comment -

          @Anne Nope I didn't, I thought this would be enough

          Show
          tubededentifrice Vincent added a comment - @Anne Nope I didn't, I thought this would be enough
          Hide
          AnneTheAgile Anne Moroney added a comment -

          @Vincent, it might be a good idea since this ticket may possibly eventually get implemented in the full version.

          Show
          AnneTheAgile Anne Moroney added a comment - @Vincent, it might be a good idea since this ticket may possibly eventually get implemented in the full version.
          Hide
          tubededentifrice Vincent added a comment -
          Show
          tubededentifrice Vincent added a comment - @Anne, done => https://jira.mongodb.org/browse/SERVER-16264
          Hide
          mghosh4@illinois.edu Mainak Ghosh added a comment -

          Hello,

          I am 4th year PhD student in UIUC working with Prof. Indranil Gupta. We have worked on this problem (wrote some code and published a paper). You can find the details of the solution in this link http://dprg.cs.uiuc.edu/docs/ICAC2015/Conference.pdf. We are currently in the process of porting the code to the new Mongo version as the original code was written v 2.2. Let me know if the solution is of interest to you and we can chat about it.

          Thanks and Regards,
          Mainak Ghosh.

          Show
          mghosh4@illinois.edu Mainak Ghosh added a comment - Hello, I am 4th year PhD student in UIUC working with Prof. Indranil Gupta. We have worked on this problem (wrote some code and published a paper). You can find the details of the solution in this link http://dprg.cs.uiuc.edu/docs/ICAC2015/Conference.pdf . We are currently in the process of porting the code to the new Mongo version as the original code was written v 2.2. Let me know if the solution is of interest to you and we can chat about it. Thanks and Regards, Mainak Ghosh.
          Hide
          jonhyman Jon Hyman added a comment - - edited

          It would also be helpful to be able to extend a shard key for increased cardinality. We are running into an issue where we shard on

          {a:1}

          and have an index on

          {a:1, b:1}

          and after two years we're starting to see some jumbo chunks. It would be nice to be able to extend the shard key to

          {a:1, b:1}

          and have the balancer now be able to split chunks on the added cardinality.

          EDIT: I see https://jira.mongodb.org/browse/SERVER-4246 exists for this. I'll vote, thanks.

          Show
          jonhyman Jon Hyman added a comment - - edited It would also be helpful to be able to extend a shard key for increased cardinality. We are running into an issue where we shard on {a:1} and have an index on {a:1, b:1} and after two years we're starting to see some jumbo chunks. It would be nice to be able to extend the shard key to {a:1, b:1} and have the balancer now be able to split chunks on the added cardinality. EDIT: I see https://jira.mongodb.org/browse/SERVER-4246 exists for this. I'll vote, thanks.

            People

            • Votes:
              19 Vote for this issue
              Watchers:
              26 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since reply:
                34 weeks, 3 days ago
                Date of 1st Reply: