[SERVER-30609] Investigate need for markThreadIdle with adaptive service executor Created: 11/Aug/17  Updated: 30/Oct/23  Resolved: 04/Jan/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 3.6.4, 3.7.1

Type: Task Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Mark Benvenuto
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File markThreadIdle.png    
Issue Links:
Backports
Related
is related to SERVER-30135 Implement PassthroughServiceExecutor ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6
Sprint: Platforms 2017-09-11, Platforms 2017-10-02, Platforms 2017-11-13, Platforms 2018-01-01, Platforms 2018-01-15
Participants:

 Description   

Each of the six runs ramps up from 0 to 500 client threads each attempting to do as many as 1000 updates per second. The system reaches CPU saturation at about 38 k/s, so of course we expect to see queuing, and to see the client open 500 connections. The six runs are respectively

  1. synchronous, 3.5.11
  2. synchronous, 3.5.11 + SERVER-28599
  3. synchronous, 3.5.11 with calls to markThreadIdle suppressed
  4. adaptive, 3.5.11
  5. adaptive, 3.5.11 + SERVER-28599
  6. adaptive, 3.5.11 with calls to markThreadIdle suppressed

Comparing runs 4, 5, and 6 we see the impact of markThreadIdle in 4 with the adaptive service executor, fixed by either change in 5 and 6.

Once the current performance issue with markThreadIdle has been addressed by SERVER-28599, we should investigate whether and where markThreadIdle should be called in the adaptive case. This should be evaluated using the tests from SERVER-16773 that led to the implementation of markThreadIdle for the synchronous case.



 Comments   
Comment by Githook User [ 01/Mar/18 ]

Author:

{'email': 'mark.benvenuto@mongodb.com', 'name': 'Mark Benvenuto', 'username': 'markbenvenuto'}

Message: SERVER-30609 Add support for markThreadIdle to adaptive service executor

(cherry picked from commit 4bc1ea069e0f9ad04af93a1442d31b87b34ddd25)
Branch: v3.6
https://github.com/mongodb/mongo/commit/1053ad1db64c99965bcb87a6a85344bed3ee8b22

Comment by Githook User [ 04/Jan/18 ]

Author:

{'name': 'Mark Benvenuto', 'username': 'markbenvenuto', 'email': 'mark.benvenuto@mongodb.com'}

Message: SERVER-30609 Add support for markThreadIdle to adaptive service executor
Branch: master
https://github.com/mongodb/mongo/commit/4bc1ea069e0f9ad04af93a1442d31b87b34ddd25

Comment by Andrew Morrow (Inactive) [ 23/Sep/17 ]

Bruce - OK, I don't mind re-opening it as we continue to investigate. I will assign it to Mark. I suspect though that if we find that we still do want to do this with the adaptive executor, that we will want a distinctly different strategy for calling MarkThreadIdle.

Comment by Henrik Edin [ 22/Sep/17 ]

Most of the problems should have gone away with the resolution of SERVER-30135 where markThreadIdle moved in to the executor instead of being in the service state machine. As the adaptive executor doesn't keep idle threads around long we have, after discussions with acm and jonathan.reams, concluded that it shouldn't be necessary to call markThreadIdle for the adaptive executor.

Generated at Thu Feb 08 04:24:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.