[SERVER-17745] Improve dirty page estimation in mmapv1 on Windows Created: 25/Mar/15  Updated: 18/Sep/15  Resolved: 09/Apr/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.0.3, 3.1.1

Type: Improvement Priority: Major - P3
Reporter: Andy Schwerin Assignee: Mark Benvenuto
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-10044 Repeated Server Crash. VirtualProtect... Closed
Backwards Compatibility: Fully Compatible
Backport Completed:
Sprint: Platform 1 04/03/15
Participants:

 Description   

On Windows, the mmapv1 storage engine underestimates the number of dirty pages. VirtualProtect failures, as in SERVER-10044, would occur less often if it kept a better estimate.



 Comments   
Comment by Githook User [ 17/Apr/15 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-17745: Improve dirty page estimation in mmapv1 on Windows

(cherry picked from commit fe358a4fd67fc48188eb062ae026001ff74082e9)
Branch: v3.0
https://github.com/mongodb/mongo/commit/cbdaff1efd3c0ba1450da45cf7ac0c6dabf62518

Comment by Githook User [ 03/Apr/15 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-17745: Improve dirty page estimation in mmapv1 on Windows
Branch: master
https://github.com/mongodb/mongo/commit/fe358a4fd67fc48188eb062ae026001ff74082e9

Comment by Mark Benvenuto [ 31/Mar/15 ]

There are two ways interesting measures for an application on Windows:

  1. For a process, we can use GetProcessMemoryInfo returns PROCESS_MEMORY_COUNTERS_EX.PagefileUsage. This will tell us how much virtual memory the machine is using. This is the VM commit charge.
  1. For the system, we can query GlobalMemoryStatusEx, and read MEMORYSTATUSEX.ullAvailPageFile. This is the remaining virtual memory on the machine. This counter tells us when the machine is about to run out of memory, and therefore VirtualProtect will fail.

I have validated these APIs work as expected under Windows 8.1, Windows 2008 R2 (7), and Windows 2008 (Vista).

We do need to set:

#define PSAPI_VERSION 1

for the pre-Windows 7 build of mongodb though.

Generated at Thu Feb 08 03:45:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.