[SERVER-21792] Performance regression with directio enabled on Windows with WiredTiger Created: 08/Dec/15 Updated: 08/Jan/24 Resolved: 15/Dec/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.2.0 |
| Fix Version/s: | 3.2.1, 3.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | Mark Benvenuto |
| Resolution: | Done | Votes: | 0 |
| Labels: | code-only | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Backport Completed: | |||||||||
| Sprint: | Platforms E (01/08/16) | ||||||||
| Participants: | |||||||||
| Description |
|
Issue Status as of Jan 05, 2016 ISSUE SUMMARY USER IMPACT WORKAROUNDS
To maintain to the performance characteristics of MongoDB 3.2.0 in later versions, start mongod with
This setting may also be configured in the mongod configuration file:
AFFECTED VERSIONS FIX VERSION Original description
3.0.7
3.2.0
Averages insert rates during initial and cache full portions of run:
|
| Comments |
| Comment by Mark Benvenuto [ 19/Jan/16 ] | |||||
|
The Yaml configuration setting is hidden:
| |||||
| Comment by Michael Dolinsky [ 18/Jan/16 ] | |||||
|
How can we configure this setting: wiredTigerEngineConfigString="direct_io=(data)" in the cfg file? | |||||
| Comment by Nick Judson [ 22/Dec/15 ] | |||||
|
@Michael Dolinsky - the original fix (which this reverts) was to fix that exact problem. Running with direct_io gives poor query performance, running without direct_io will cause all the system memory to be consumed. In 3.2, this wasn't the case (for my workload). I had good query performance and constant memory usage. I didn't have any regressions as described above. Now it sounds like the caching strategy for windows may be revisited for the next major release. | |||||
| Comment by Michael Dolinsky [ 20/Dec/15 ] | |||||
|
So with 3.2.1 if there are a lot of writes (more then the system can handle) essentially the system cache will take up all memory and would cause an OS issue / MongoDB issue | |||||
| Comment by Daniel Pasette (Inactive) [ 17/Dec/15 ] | |||||
|
The nightly development releases can be found here: From there, you have to select your os version and mongodb version | |||||
| Comment by Michael Dolinsky [ 16/Dec/15 ] | |||||
|
Where can we download the 3.2.1 bits to test this? | |||||
| Comment by Mark Benvenuto [ 16/Dec/15 ] | |||||
|
In 3.2.1, the old behavior which reduces memory usage, but has a performance regression in some scenarios can be achieved as followed: --wiredTigerEngineConfigString="direct_io=(data)". | |||||
| Comment by Mark Benvenuto [ 15/Dec/15 ] | |||||
|
Master: https://github.com/mongodb/mongo/commit/439a56d7af3ce4bad983f5829b3485bb0af7f6c3 v3.2: https://github.com/mongodb/mongo/commit/c7b065227470a27c40c45f07a8c967b7aa7af9db | |||||
| Comment by Ramon Fernandez Marina [ 10/Dec/15 ] | |||||
|
stephen.jannin@sgcib.com, your statement is incorrect:
| |||||
| Comment by Stephen JANNIN [ 10/Dec/15 ] | |||||
|
Warning, if you use direct_io, you have very poor performances on windows. | |||||
| Comment by Bruce Lucas (Inactive) [ 09/Dec/15 ] | |||||
|
A run with Windows performance counters, with default settings (direct i/o enabled).
Theory: when memory usage reaches a certain point, the o/s starts spilling memory to disk proactively (not in response to memory pressure as such), and for some reason tbd this in itself has a negative performance impact. Note that the following do not appear to be the reason for the negative performance impact:
Why do we not see this effect with direct i/o disabled? We're doing a run now with direct i/o disabled to see how the stats differ. Theory: as long as there is dirty file data in the file cache Windows preferentially writes that instead of paging process memory out, so by in effect disabling the file cache we've given Windows the green light to page process memory out, and that for reasons tbd is having a negative performance impact. | |||||
| Comment by Githook User [ 09/Dec/15 ] | |||||
|
Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Signed-off-by: kay <kay.kim@10gen.com> | |||||
| Comment by Bruce Lucas (Inactive) [ 08/Dec/15 ] | |||||
|
This seems to be a result using direct i/o, which was enabled for Windows in rc3 under
Adding this result to the table above:
Direct i/o can be disabled by specifying --wiredTigerEngineConfigString "direct_io=()" (or equivalent config file option), and this should be a useful workaround for now. |