Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-48677

Create reshardCollection command on config server

    • Fully Compatible
    • Sharding 2020-07-13, Sharding 2020-06-29, Sharding 2020-07-27

      • Add required input parameter validation
      • Change router command to contact config server
      • Add unit tests
      • Add integration tests

      Step One: Creating the command files.

      1. Take the schema defined in the technical design for the project. Create a new IDL definition for the config server command in the same file as the router command.
      2. Create a corresponding C++ command file in src/mongo/db/s/config/configsvr_reshard_collection_cmd.cpp. This command may just return true for now.
      3. Change the router command so that we call the config server command, and assert that the config server command returns true.
      4. Make sure your javascript test works.

      Step Two: Adding input validation.

      • Make sure your input validation is copied over from the router command.
      • Validate that the zones parameter exists if any entries exist in config.tags, a collection on the config server that holds entries for all of the current zones/tags that exist. You should see if any functions exist for this in the ShardingCatalogManager class. If not, you should make one.

      Step Three: Creating parsers for the zones

      • Create zone_util.h and zone_util.cpp files in src/db/s. You can use range_deletion_util as inspiration. The following logic should be in zone_util separated out into functions. Any functionality that specifically interacts with the underlying catalog metadata that's on disk should be in the ShardingCatalogManager. Any functionality that doesn't interact with the underlying metadata should be in zone_util. so use your best judgment to make functions either on the catalog manager or inside zone_util. Keep in mind that it will be an easier sell to reviewers to put things into zone_util, however.
      • Verify that zones parse correctly using the TagsType::fromBSON function. In order to do this, you may need to concatenate the namespace to the objects.
      • For each zone, verify that the zone name corresponds to a zone name that already exists in config.tags. Look again in ShardingCatalogManager for this functionality. These zones are completely new, so you don't have to verify any ranges against ranges that already exist in config.tags. However, we must make sure that resharding isn't generating new zones, and that we're using zones that already exist in the system. This is because defining a new zone would also require us to define shards that exist for the zone, and the schema we decided on doesn't allow to define shards along with their zones.
      • For the zones as a whole, verify that there aren't any overlapping ranges. We don't need to verify if the range as a whole is covered – if there are holes in the range, chunks will be created without zones.
      • Place calls to these functions inside the config server command.

      Step Four: Creating parsers for _presetReshardedChunks

      • Create resharding_util.h and resharding_util.cpp files in src/db/s/. We will use this area to maintain common helper functionality for the resharding process. The following logic will be maintained there.
      • Verify that the chunks inserted parse correctly with regard to a valid shard id and valid BSON objects for the min and max.
      • Verify that the array of chunks is valid and covers the entire range for the collection. You may be able to use the logic in constructShardVersionMap to help with this. What you're going to make in the end will be much more lightweight than that example... all you're going to want to do is iterate over an ordered list of {{ChunkRange}}s to verify their correctness.

      Step Four: Creating unit tests

      • Create unit tests to test resharding_util and zone_util.

      Step Five: Creating integration tests

      • Create tests verifying that what you've created works well so far.

            kshitij.gupta@mongodb.com Kshitij Gupta
            blake.oler@mongodb.com Blake Oler
            0 Vote for this issue
            3 Start watching this issue