[SERVER-24352] configs initialize sharding awareness on sharding-unaware shards on transition to primary Created: 01/Jun/16  Updated: 13/Aug/16  Resolved: 27/Jul/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.3.6
Fix Version/s: 3.3.11

Type: Task Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Esha Maharishi (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-22660 OpObserver on config server for inser... Closed
is related to SERVER-22663 Make --shardsvr required for a mongod... Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 15 (06/03/16), Sharding 16 (06/24/16), Sharding 17 (07/15/16), Sharding 18 (08/05/16)
Participants:

 Description   

Add a class that can run a BackgroundJob on the config servers to initialize sharding awareness on any shards in config.shards that are not marked as sharding aware (as represented by a flag field in config.shards).

The BackgroundJob can be kicked off on transition to primary or by an OpObserver watching for inserts to config.shards.

While the cluster upgrade is in progress:

  • If a non-3.4 shard is added by an old mongos OR the new _configsvrAddShard process, the shardIdentity document will be inserted into the shard and remain latent until the shard is upgraded to 3.4 and restarted with --shardsvr, at which point the shard will become sharding aware. The configs will incorrectly but harmlessly believe the shard is sharding aware (that is, the shard will be marked as "sharding aware" in config.shards). This is because the insert of the shardIdentity document won't have been impeded by an OpObserver on the shard checking for --shardsvr.
  • If a 3.4 shard is added through the new _configsvrAddShard process, all works as designed for a 3.4 cluster.
  • If a 3.4 shard started with --shardsvr is added by an old mongos, the OpObserver on the configs will asynchronously initialize sharding awareness on it by kicking off the BackgroundJob.
  • If a 3.4 mongod without --shardsvr is added by an old mongos, the OpObserver will kick off the asynchronous BackgroundJob, but the mongod will reject the shardIdentity document. The old mongos will believe the shard has been added, but all sharding requests the mongos sends to it will be rejected by the shard. TODO: Should the BackgroundJob remove the shard's entry from config.shards, so that the old mongos stops trying to communicate with it as soon as it reloads the ShardRegistry?


 Comments   
Comment by Githook User [ 27/Jul/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-24352 configs initialize sharding awareness on sharding-unaware shards on transition to primary
Branch: master
https://github.com/mongodb/mongo/commit/e3068c25d388166433fe2120d835375e7c9f76ad

Comment by Esha Maharishi (Inactive) [ 01/Jun/16 ]

Sounds good, maybe a method can be added to CatalogManager to do the check in config.shards and that uses the executorForAddShard to do the shardIdentity insert.

Comment by Spencer Brody (Inactive) [ 01/Jun/16 ]

I don't think this is done as a literal BackgroundJob subclass. I think you just have a function that is called during config server transition to primary, and the OpObserver just schedules that function to run in a TaskExecutor.

Comment by Esha Maharishi (Inactive) [ 01/Jun/16 ]

Resolution for the TODO: we decided that the insert of the shardIdentity document should not fail on the shard, and that the shard can simply reject sharding requests until it is restarted with --shardsvr. This is to allow for manual modification (e.g., delete and re-insert) of the shardIdentity document during things like backup-restore.

Comment by Esha Maharishi (Inactive) [ 01/Jun/16 ]

renctan, spencer, please confirm if the behavior in the TODO in the last bullet point is what we want?

Generated at Thu Feb 08 04:06:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.