[SERVER-25673] Remove ChunkManager use in the MigrationManager Created: 17/Aug/16  Updated: 05/Apr/17  Resolved: 17/Jan/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 3.5.2

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Dianna Hohensee (Inactive)
Resolution: Done Votes: 0
Labels: ds-neweng-2016, neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-27510 Compare only epochs instead of full c... Closed
Related
related to SERVER-27632 Replace 'shardVersion' field in split... Closed
is related to SERVER-25527 Send the version of the chunk being m... Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2017-01-02
Participants:

 Description   

We load+refresh the ChunkManager when the Balancer finds chunks to balance. And then we do it again in the MigrationManager to get collection version information to send with the moveChunk command

1)

  • Add a chunkVersion and collectionVersion field in MigrateInfo.
  • The balancer chunk selection policy represents chunks it selects to be balanced as MigrateInfo objects, which are sent to the Balancer to schedule.
  • Then update MigrationType to serialize/parse for chunkVersion and collectionVersion, so config.migrations documents include them and MigrationManager recovery can build correct MigrateInfo objects to reschedule. (go look at type_migration.h/cpp on github for this – the class was checked in with version serialization/parsing, but it had to be removed later, so the code's already there in the file histories. Same for scoped_migration_request.h/cpp).
  • Then, instead of loading up the ChunkManager again, just get the collection version from the MigrateInfo that's used to schedule a specific migration.

2)

  • Pull out the ChunkManager code from the MigrationManager, as it is no longer needed if the MigrateInfo has both the chunk and collection versions.
  • This looks like it will require some changes to manual moveChunk commands coming through from the mongos. Either the Balancer::rebalanceSingleChunk and Balancer::moveSingleChunk must load the ChunkManager now, or (I think better) the mongos should send chunkVersion and collectionVersion in the _configsvrMoveChunk command through to these balancer functions.


 Comments   
Comment by Dianna Hohensee (Inactive) [ 17/Jan/17 ]

Recommitted after removing shard_does_not_hang_on_bad_config_server.js as part of SERVER-27625, which is in progress. The test is no longer significant because shard servers are started up sharding aware now and do not become sharding aware from commands. So the test was deleted, rather than fixing it for this commit and then deleting it shortly with the main SERVER-27625 commit.

Comment by Githook User [ 17/Jan/17 ]

Author:

{u'username': u'DiannaHohensee', u'name': u'Dianna Hohensee', u'email': u'dianna.hohensee@10gen.com'}

Message: SERVER-25673 Remove redundant ChunkManager access in MigrationManager
Branch: master
https://github.com/mongodb/mongo/commit/fdec6ed545c0045646c7cca33eb4094385bc9429

Comment by Githook User [ 11/Jan/17 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Revert "SERVER-25673 Remove redundant ChunkManager access in MigrationManager"

This reverts commit d779870e5d4744cbcc402cba2a77f8d892bed0ef.
Branch: master
https://github.com/mongodb/mongo/commit/c920eb38186c6122fed326ba341e05d7e02ffe37

Comment by Mathias Stearn [ 11/Jan/17 ]

Broke noPassthrough/shard_does_not_hang_on_bad_config_server.js: https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_noPassthrough_d779870e5d4744cbcc402cba2a77f8d892bed0ef_17_01_11_21_52_35

Comment by Githook User [ 11/Jan/17 ]

Author:

{u'username': u'DiannaHohensee', u'name': u'Dianna Hohensee', u'email': u'dianna.hohensee@10gen.com'}

Message: SERVER-25673 Remove redundant ChunkManager access in MigrationManager
Branch: master
https://github.com/mongodb/mongo/commit/d779870e5d4744cbcc402cba2a77f8d892bed0ef

Comment by Dianna Hohensee (Inactive) [ 06/Jan/17 ]

As tangential cleanup:

  • will remove 'shardVersion' field sent in shard moveChunk – we only use the epoch, and we send an 'epoch' field already.
  • will remove 'chunkVersion', which isn't needed. moveChunk already includes a 'shardVersion' field, from which the epoch can be obtained. – this depends on SERVER-27510, to change the moveChunk command check to epoch only, rather than specific ChunkVersion.
Comment by Dianna Hohensee (Inactive) [ 06/Jan/17 ]

Already incidentally completed since this was created:

  • MigrateInfo has ChunkVersion
  • The balancer chunk selection policy returns MigrateInfo objects.

Still outstanding:

  • MigrateInfo must include the chunk's shard's expected collectionVersion (currently passed in 'shardVersion' field of moveChunk). This can come from the balancer chunk selection policy when it accesses the ChunkManager anyway.
  • MigrationManager should use MigrateInfo.version (ChunkVersion), rather than getting the chunk from ChunkManager and then getting the version from that Chunk object.
  • Clean up manual moveChunk commands. Mongos does not need to send the chunk's ChunkVersion or expected shardVersion. The config server will acquire this information, given the chunk range.
Generated at Thu Feb 08 04:09:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.