[SERVER-36783] Run the secondary_reads_passthrough task of enterprise-rhel-62-64-bit-inmem on rhel62-large Created: 21/Aug/18  Updated: 29/Oct/23  Resolved: 27/Aug/18

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.6.9, 4.0.3, 4.1.3

Type: Task Priority: Major - P3
Reporter: Yves Duhem Assignee: Max Hirschhorn
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-34384 Passthrough test for secondary reads ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6
Sprint: TIG 2018-09-10
Participants:
Linked BF Score: 30

 Description   

The secondary_reads_passthrough task on enterprise-rhel-62-64-bit-inmem runs 20 mongods (4 jobs with a 5 node cluster) and occasionally the host runs out of memory, leading to one of the mongod being killed.

The task should be moved to run on a rhel62-large distro instead of a rhel62-small currently.



 Comments   
Comment by Githook User [ 25/Oct/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-36783 Run secondary_reads_passthrough with more memory headroom.

Changes the Enterprise RHEL 6.2 (inMemory) build variant to run the
secondary_reads_passthrough task with --jobs=5 on the rhel62-large
distro.

(cherry picked from commit 0eb340d54510584b1d1c927e713a5afe0822feaf)
Branch: v3.6
https://github.com/mongodb/mongo/commit/365ae29ee335257398159f1b9f1f9767198fe607

Comment by Githook User [ 30/Aug/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-36783 Run secondary_reads_passthrough with more memory headroom.

Changes the Enterprise RHEL 6.2 (inMemory) build variant to run the
secondary_reads_passthrough task with --jobs=5 on the rhel62-large
distro.

(cherry picked from commit 0eb340d54510584b1d1c927e713a5afe0822feaf)
Branch: v4.0
https://github.com/mongodb/mongo/commit/51d917ad2b9657672f5b7040983b32623ac0d0f0

Comment by Githook User [ 27/Aug/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-36783 Run secondary_reads_passthrough with more memory headroom.

Changes the Enterprise RHEL 6.2 (inMemory) build variant to run the
secondary_reads_passthrough task with --jobs=5 on the rhel62-large
distro.
Branch: master
https://github.com/mongodb/mongo/commit/0eb340d54510584b1d1c927e713a5afe0822feaf

Comment by Max Hirschhorn [ 23/Aug/18 ]

Thanks for the clarification yves.duhem. Focusing on the amount of memory available for each resmoke.py job is a great point. We had to make similar decisions for SERVER-29353, SERVER-29355, and SERVER-29356 when moving some build variants over to the large instance types.

Comment by Yves Duhem [ 23/Aug/18 ]

max.hirschhorn, we still want to increase the instance size but, as robert.guo pointed out, the ratio of RAM to CPU is the same on the rhel62-large. So the ticket will also involve limiting the number of jobs the task will run on this build variant.

Comment by Max Hirschhorn [ 23/Aug/18 ]

cristopher.stauffer, yves.duhem, why don't we want to increase the instance size? My recollection from the linked BF ticket is that each mongod using ~500MB of memory which seems reasonable to me.

Comment by Cristopher Stauffer [ 23/Aug/18 ]

Based on our conversation during story pointing, this should turn into a research ticket to identify the correct approach for resolving this issue.

Generated at Thu Feb 08 04:44:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.