[SERVER-82592] Remove _kChunkOpLock lock Created: 30/Oct/23  Updated: 06/Nov/23

Status: Backlog
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor - P4
Reporter: Marcos José Grillo Ramirez Assignee: Backlog - Catalog and Routing
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-82408 Deprecate ConfigsvrSetAllowMigration ... Blocked
Assigned Teams:
Catalog and Routing
Participants:

 Description   

Currently chunk operations are serialized by the _kChunkOpLock, which is taken whenever there is a metadata change in the config.chunks collection in the config server (for example split, merge, chunk migrations, refine collection shard key, among much others). In the case of split/merge/migration, this has the unfortunate side effect of serializing operations that belongs to complete different collections, even though, this is not actually necessary.

We could increase the level of parallelization for DDL operations by refining the synchronization to a more fine-grained degree, for example, at the chunk level. This could be achieved by first making all operations under the lock to be performed in a transaction, and then, as part of the transaction we could search for the chunk with the collection version, and then doing a dummy write at the document level (for example, increasing a dummy counter). The result of this would be having ConflictingOperationInProgress errors when trying to commit two concurrent operations on the same collection, which could be retried by an upper layer (i.e. by DDLCoordinators).


Generated at Thu Feb 08 06:49:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.