[SERVER-28169] Set eviction=(threads_min=1) in Microbenchmarks testing Created: 02/Mar/17  Updated: 31/Mar/17  Resolved: 06/Mar/17

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: None
Fix Version/s: 3.2.13, 3.4.4, 3.5.4

Type: Bug Priority: Major - P3
Reporter: Henrik Ingo (Inactive) Assignee: Henrik Ingo (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-28179 inMemory: fassert when using --inMemo... Closed
is related to SERVER-28026 Disable auto-tuning of WiredTiger evi... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4, v3.2, v3.0
Sprint: Performance 2017-03-27
Participants:
Linked BF Score: 0

 Description   

SERVER-28026 changed a wiredTiger / inMemory internal setting

eviction=(threads_min=)

...from 0 to 4. This seems to interfere with results for the Microbenchmarks suite and in any case we generally turn off such background processes in Microbenchmarks. I will set this back to zero using --inMemoryEngineConfigString.



 Comments   
Comment by Githook User [ 31/Mar/17 ]

Author:

{u'username': u'henrikingo', u'name': u'Henrik Ingo', u'email': u'henrik.ingo@mongodb.com'}

Message: SERVER-28169 Set eviction=(threads_min=1) in Microbenchmarks testing

Undoing the effect of 5a3ecdc731673c82cc841eaecf2bc59067c067be for microbenchmarks.

(cherry picked from commit b078623660b1f4582cbb90998f8e6c252d3bd7b8)
Branch: v3.4
https://github.com/mongodb/mongo/commit/4a3ecd3dd5ff54b478eb7d4b5c28114976c6995b

Comment by Githook User [ 24/Mar/17 ]

Author:

{u'username': u'henrikingo', u'name': u'Henrik Ingo', u'email': u'henrik.ingo@mongodb.com'}

Message: SERVER-28169 Set eviction=(threads_min=1) in Microbenchmarks testing

Undoing the effect of 5a3ecdc731673c82cc841eaecf2bc59067c067be for microbenchmarks.

(cherry picked from commit b078623660b1f4582cbb90998f8e6c252d3bd7b8)
Branch: v3.2
https://github.com/mongodb/mongo/commit/ca880118d629ceeec952cc526ec71ddc08aea739

Comment by Henrik Ingo (Inactive) [ 06/Mar/17 ]

Master done. Needs backporting once 3.4 is unfrozen.

Comment by Githook User [ 06/Mar/17 ]

Author:

{u'username': u'henrikingo', u'name': u'Henrik Ingo', u'email': u'henrik.ingo@mongodb.com'}

Message: SERVER-28169 Set eviction=(threads_min=1) in Microbenchmarks testing

Undoing the effect of 5a3ecdc731673c82cc841eaecf2bc59067c067be for microbenchmarks.
Branch: master
https://github.com/mongodb/mongo/commit/b078623660b1f4582cbb90998f8e6c252d3bd7b8

Comment by Henrik Ingo (Inactive) [ 03/Mar/17 ]

Thanks michael.cahill

I realized this, but my first instinct is to just try to revert the recent change back to what it was.

It's a good rule in performance testing to only do one change at a time, and for now the focus is simply to identify what has changed in our results, around Feb 21 or so, and how can we get back to where we were before. A followup action could indeed be to set threads_max lower as well.

Second, while I understand how the new default of threads_min would in general contribute to more stable performance behavior, I'm suspecting the microbenchmarks behave differently from real world workloads here. They're mostly just a few hundred rows or so, and in this case we're talking readonly tests. So the theory is that just getting back to how things were, will in fact get rid of the occasional drops we're now seeing. If we first observe that (for some weeks), then we could indeed tae a further step of locking both min_threads and max_threads at 2, or even 1. (In microbenchmarks we generally just turn off everything, but it might not be feasible here to use 0. As you remember, we don't restart mongod between tests, so some eviction activity might currently be necessary.)

Comment by Michael Cahill (Inactive) [ 03/Mar/17 ]

I think it's worth clarifying what this setting controls.

The WiredTiger and inMemory storage engines have always had the setting eviction=(threads_max=4). That means when the cache becomes full and eviction starts running, up to 4 background threads will be started.

As SERVER-28026 explains, some WiredTiger work in WT-2898 changes when those background threads are started. We saw higher variation in test runs without that configuration change because sometimes background threads would start and sometimes they wouldn't. That is what led to changing the defaults to eviction=(threads_min=4,threads_max=4) – i.e., always start 4 eviction threads.

I understand wanting to minimize the impact of background threads on the microbenchmarks, but I suggest that if less threads improves the test results, we try something like eviction=(threads_min=2,threads_max=2). That is, keep the min/max settings equal but make them both smaller.

Generated at Thu Feb 08 04:17:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.