[SERVER-27681] Better concurrency control on shard metadata refresh Created: 13/Jan/17  Updated: 04/Sep/17  Resolved: 20/Mar/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.2.12, 3.4.2, 3.5.1
Fix Version/s: 3.4.4, 3.5.5

Type: Improvement Priority: Major - P3
Reporter: Randolph Tan Assignee: Kaloian Manassiev
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-20854 Don't refresh database and collection... Closed
Duplicate
is duplicated by SERVER-28386 Skip invalidated collections in Catal... Closed
is duplicated by SERVER-27475 mongos should request an update only ... Closed
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4
Sprint: Sharding 2017-03-27
Participants:
Linked BF Score: 0

 Description   

Currently, the shard limits the number of threads refreshing the config metadata here. This limit is very crude and does not differentiate refresh for different namespaces. It is possible that the all the ticket holders are trying to refresh the same collection. In this case, it limits parallelism while doing wasted work.



 Comments   
Comment by Githook User [ 12/Apr/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-27681 Use CatalogCache to load metadata on shards

This change gets rid of the MetadataLoader in place of using the
CatalogCache for loading chunk metadata on shards. This ensures that the
same concurrency control and rate limiting applies on mongos and mongod
and obviates the need for having different kinds of ChunkDiffer.

Also removes the duplicated ShardKeyPattern functionality from
CollectionMetadata in place of using the one from ShardKeyPattern.

(cherry picked from commit 3e53f0ce9cdff926473276a34a351972ea963a57)
(cherry picked from commit f35cb0e4fcc26e5c8950fe75a8ff46aa811f2e56)
(cherry picked from commit d8b0d028e3895b32b86f3e6878d4bc34dd8c7fc8)
Branch: v3.4
https://github.com/mongodb/mongo/commit/4044079b46f8c55ceb6b1df121330bf83b81a8b3

Comment by Githook User [ 22/Mar/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-27681 Remove duplicated ShardKeyPattern functionality from CollectionMetadata
Branch: master
https://github.com/mongodb/mongo/commit/d8b0d028e3895b32b86f3e6878d4bc34dd8c7fc8

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-27681 Fix broken OSX compile due to unused variable
Branch: master
https://github.com/mongodb/mongo/commit/f35cb0e4fcc26e5c8950fe75a8ff46aa811f2e56

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-27681 Use CatalogCache to load metadata on shards

This change gets rid of the MetadataLoader in place of using the
CatalogCache for loading chunk metadata on shards. This ensures that the
same concurrency control and rate limiting applies on mongos and mongod
and obviates the need for having different kinds of ChunkDiffer.
Branch: master
https://github.com/mongodb/mongo/commit/3e53f0ce9cdff926473276a34a351972ea963a57

Comment by John Page [ 19/Jan/17 ]

See also #SERVER-20854 which seems to be similar but for MongoS.

Generated at Thu Feb 08 04:15:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.