Details
-
Task
-
Resolution: Duplicate
-
Major - P3
-
None
-
None
-
None
-
Sharding 2018-06-04, Sharding 2018-06-18, Sharding 2018-07-02
Description
With the new autosplitting design, a thread pool is used to asynchronously execute chunk splits. This can lead to scenarios where, for example, a request to split one chunk is submitted, more writes come in, and another request to split the same chunk is submitted before the first is executed. Those two tasks could then either be executed concurrently or sequentially. This can lead to:
- Potentially duplicated work, if two threads concurrently process conflicting work corresponding to the same chunk, and only one succeeds
- If you’re loading a lot into a chunk that’s currently being split you could end up with the task list growing large until the chunk is split, followed by a bunch of useless work when the chunk is finally split
- Almost unbounded growth of the task queue, if your splitting can't keep up with writes
One solution to this is to keep track of which chunks have split tasks currently pending/executing, and when a new task is submitted it should be checked against the list of pending tasks. If it corresponds to the same chunk as a currently executing task, it should return silently without doing any work.
Attachments
Issue Links
- depends on
-
SERVER-35612 Implement ChunkWritesTracker to track bytes written to a chunk
-
- Closed
-
- duplicates
-
SERVER-35612 Implement ChunkWritesTracker to track bytes written to a chunk
-
- Closed
-