[SERVER-50290] Enforce stronger limits on task splitting Created: 13/Aug/20 Updated: 29/Oct/23 Resolved: 13/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.1, 4.7.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | David Bradford (Inactive) | Assignee: | David Bradford (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Backport Requested: |
v4.4
|
||||
| Sprint: | DAG 2020-08-24 | ||||
| Participants: | |||||
| Description |
|
We have found that splitting tasks too aggressively we can create a lot of extra work in setup and teardown of the tests, which causes us to split tasks even more, which leads to more setup and teardown. By limiting the number of sub-tasks we create, we can contain the growth of the setup/teardown overhead. In some exploration, it appears that a max split of 3 gives us similar makespans as we currently have, which minimizing the overhead of additional tasks. As a server engineer, AC:
|
| Comments |
| Comment by Harris Hoke (Inactive) [ 19/Aug/20 ] |
|
Just for the last 24 hours, we didn’t even hit max_hosts on either rhel62 distro, much less maxing out for those long nightly plateaus we’ve come to expect. With one day of this I would be worried it was just a fluke due to fluctuations in volume, but two days is extremely promising to me! |
| Comment by Githook User [ 13/Aug/20 ] |
|
Author: {'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}Message: (cherry picked from commit 53b690294d160a6e1cd0f4a004cb55c4835a48b8) |
| Comment by Githook User [ 13/Aug/20 ] |
|
Author: {'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}Message: |