[SERVER-33147] Limit the -j/num_jobs_available for high core ARM CI servers Created: 06/Feb/18  Updated: 29/Oct/23  Resolved: 12/Feb/18

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.4.14, 3.6.3, 3.7.2

Type: Task Priority: Major - P3
Reporter: Zakhar Kleyman Assignee: Zakhar Kleyman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6, v3.4
Participants:

 Description   

We use $(grep -c ^processor /proc/cpuinfo) as the -j/num_jobs_available value for ubuntu1604-arm64 distros right now.
As we're about to get new high core build ARM servers, we should limit the num_jobs_available because some test might fail when the value is too high (nobody runs 96 mongod instances on a single server in a real setup).

We should do it in such way that this would work on both new and old servers.



 Comments   
Comment by Githook User [ 12/Feb/18 ]

Author:

{'email': 'zakhar.kleyman@mongodb.com', 'name': 'Zakhar Kleyman', 'username': 'zakhark'}

Message: SERVER-33147 limit the max_jobs for ubuntu1604-arm64 distros
Branch: v3.6
https://github.com/mongodb/mongo/commit/5adc0e0b4be022471b9b1803f109d7afac675d32

Comment by Githook User [ 12/Feb/18 ]

Author:

{'email': 'zakhar.kleyman@mongodb.com', 'name': 'Zakhar Kleyman', 'username': 'zakhark'}

Message: SERVER-33147 limit the max_jobs for ubuntu1604-arm64 distros
Branch: v3.4
https://github.com/mongodb/mongo/commit/9aea96e4bfa630b36ba39f947de8786b18a1b0a8

Comment by Githook User [ 12/Feb/18 ]

Author:

{'email': 'zakhar.kleyman@mongodb.com', 'name': 'Zakhar Kleyman', 'username': 'zakhark'}

Message: SERVER-33147 limit the max_jobs for ubuntu1604-arm64 distros
Branch: master
https://github.com/mongodb/mongo/commit/ba23a370d311bec5b66c1945decaf1017b2f1a1d

Comment by Zakhar Kleyman [ 06/Feb/18 ]

Thanks, max.hirschhorn

I was thinking of doing something like
"$(( $(grep -c ^processor /proc/cpuinfo) > 8 ? 8 : $(grep -c ^processor /proc/cpuinfo) ))"
but I'll look into ${max_jobs} expansion.

Comment by Max Hirschhorn [ 06/Feb/18 ]

zakhar.kleyman, you could consider setting the ${max_jobs} expansion as a way to define a hard limit on the maximum number of tests that resmoke.py will run concurrently on the ARM machine.

https://github.com/mongodb/mongo/blob/84ec77665ce351f5eac8a19876cfe48d326e0ca2/etc/evergreen.yml#L631-L635

Generated at Thu Feb 08 04:32:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.