[SERVER-45201] Implicit collection creation from createIndexes can stall replication on secondaries by circumventing thread pool on primary Created: 17/Dec/19 Updated: 29/Oct/23 Resolved: 14/Jan/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 4.3.3 |
| Fix Version/s: | 4.3.3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Benety Goh |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2020-01-13, Execution Team 2020-01-27, Execution Team 2019-12-30 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Linked BF Score: | 43 | ||||||||||||||||||||||||||||||||
| Description |
|
Implicit collection creation bypasses the IndexBuildsCoordinator thread pool on the primary and replicates "startIndexBuild" oplog entries directly. The thread pool size is 10 on both primaries and secondaries, and is used to limit resource usage. Each additional index build will block until a thread resource is available. This is problematic in the following scenario:
This results in a deadlock, which stalls replication on the secondary |
| Comments |
| Comment by Githook User [ 14/Jan/20 ] |
|
Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}Message: |
| Comment by Githook User [ 13/Jan/20 ] |
|
Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}Message: These tests rely on fail points that work only with index builds on non-empty collections. |
| Comment by Benety Goh [ 18/Dec/19 ] |
|
There's a potential for optimizing the empty collection case in this part of IndexBuildsCorodinator::_filterSpecsAndRegisterBuild(). This will also improve the behavior for the createIndexes command on a primary node on an empty collection. |
| Comment by Louis Williams [ 17/Dec/19 ] |
|
One approach could be to use the same logic on the secondary to decide whether or not to circumvent the thread pool on the primary. |