[SERVER-47428] Move concurrency_sharded* Evergreen tasks to -large distros Created: 09/Apr/20 Updated: 29/Oct/23 Resolved: 15/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.0-rc2, 4.7.0, 4.2.10 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Max Hirschhorn | Assignee: | Max Hirschhorn |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | sharding-4.4-stabilization | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Backport Requested: |
v4.4, v4.2
|
||||||||
| Sprint: | Sharding 2020-04-20 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 18 | ||||||||
| Description |
|
The concurrency_sharded*.yml test suites run against a 2-shard, 3-node replica set shard cluster (sometimes with a 1-node CSRS and sometimes with a 3-node CSRS) with 2 mongos processes. They currently run on the -small distros, which typically correspond to c4.xlarge and have 7.5GB of memory. The 6+ mongod processes tend not to leave much headroom for the other MongoDB processes, resmoke.py buffering log messages, or gdb attaching to a process. We have recently been seeing the OOM killer killing processes on the machine and causing the Evergreen task to fail. Moving to the -large distros, which typically correspond to c4.4xlarge and have 30GB of memory, should address these failures. This involves doing the following:
|
| Comments |
| Comment by Githook User [ 10/Aug/20 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: Not all build variants run the concurrency* tasks with the "large" task (cherry picked from commit 6202c52a254ced5238eaf7f77117f7c4865aed56) |
| Comment by Githook User [ 15/Apr/20 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: Not all build variants run the concurrency* tasks with the "large" task (cherry picked from commit 6202c52a254ced5238eaf7f77117f7c4865aed56) |
| Comment by Githook User [ 15/Apr/20 ] |
|
Author: {'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}Message: Not all build variants run the concurrency* tasks with the "large" task |