[SERVER-14992] Query for Windows 7 File Allocation Fix, and other hotfixes Created: 21/Aug/14  Updated: 15/Nov/21  Resolved: 09/Jan/15

Status: Closed
Project: Core Server
Component/s: Internal Code
Affects Version/s: None
Fix Version/s: 2.6.9, 2.8.0-rc5

Type: Bug Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Mark Benvenuto
Resolution: Done Votes: 0
Labels: community-team, pull-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-4639 Recommend that Microsoft KB 2731284 b... Closed
Duplicate
is duplicated by SERVER-16634 CLONE - Patch to speed up rs.initiate... Closed
Related
is related to SERVER-24395 Do not display startup warnings when ... Closed
is related to SERVER-12401 Improve the memory-mapped files flush... Closed
Backwards Compatibility: Fully Compatible
Operating System: Windows
Backport Completed:
Sprint: Server 2.7.6
Participants:

 Description   

We need to query Win32_QuickFixEngineering in WMI to get information about the installed KB articles

Here is an example for the recent sync performance fix:
http://support.microsoft.com/kb/2981965

> Get-WmiObject -query 'Select * From Win32_QuickFixEngineering'
 
Source        Description      HotFixID      InstalledBy          InstalledOn
------        -----------      --------      -----------          -----------
WIN-AB4P31... Hotfix           KB2981965     NT AUTHORITY\SYSTEM  8/21/2014 12:00:00 AM

References:
http://msdn.microsoft.com/en-us/library/aa394558%28v=vs.85%29.aspx
http://msdn.microsoft.com/en-us/library/aa390423%28v=vs.85%29.aspx
http://msdn.microsoft.com/en-us/library/aa394391%28v=vs.85%29.aspx



 Comments   
Comment by Githook User [ 05/Mar/15 ]

Author:

{u'username': u'agherardi', u'name': u'agherardi', u'email': u'alessandro.gherardi@yahoo.com'}

Message: SERVER-14992: Checking ntfs.sys version to figure out if it's necessary to zero out data files.

Closes #924

Signed-off-by: Ramon Fernandez <ramon.fernandez@mongodb.com>
Branch: v2.6
https://github.com/mongodb/mongo/commit/77fb852fb232c846b9a22a9a72c262639df9c47c

Comment by Ramon Fernandez Marina [ 12/Feb/15 ]

Thanks for the pull request agherardi. I'm tentatively approving the backport for 2.6.9 pending a full test run.

Comment by Mark Benvenuto [ 09/Jan/15 ]

Docs:

We should ensure that we recommend customers install Microsoft KB 2731284 on Windows 7 & Windows 2008 R2 machines to get better file allocation performance for MMapV1 databases.

http://support.microsoft.com/kb/2731284

Comment by Mark Benvenuto [ 09/Jan/15 ]

Yes, it is in 2.8. It will part of the next 2.8 rc release.

Comment by Alessandro Gherardi [ 09/Jan/15 ]

Thanks, Mark.

Is this patch going to make it into 2.8?

Comment by Githook User [ 09/Jan/15 ]

Author:

{u'username': u'agherardi', u'name': u'agherardi', u'email': u'alessandro.gherardi@yahoo.com'}

Message: SERVER-14992: Checking ntfs.sys version to figure out if it's necessary to zero out data files.

Closes #894

Signed-off-by: Mark Benvenuto <mark.benvenuto@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/aee446a97d7c3d7b6129348de6957d8e178864b2

Comment by Alessandro Gherardi [ 07/Jan/15 ]

I updated the pull request per CR comments.

Can the pull request be merged now?

Comment by Alessandro Gherardi [ 06/Jan/15 ]

Mark - Any updates?

Thanks,
Alessandro

Comment by Alessandro Gherardi [ 01/Jan/15 ]

Created pull request:
https://github.com/mongodb/mongo/pull/894

Comment by Alessandro Gherardi [ 30/Dec/14 ]

Hi Mark,
I made the changes you suggested. Noticed that I used SHGetSpecialFolderPath because SHGetKnownFolderPath requires an NTDDI_VERSION of at least NTDDI_VISTA (0x06000000) but the build script sets NTDDI_VERSION to NTDDI_WS03SP2 (0x05020200).

Thanks,
Alessandro

Comment by Mark Benvenuto [ 30/Dec/14 ]

Your change looks pretty good.

I have a few minor suggestions though:

  1. Instead of char *verData = new char[verSize]; use boost::scoped_array<char> verData = new char[verSize];. This way you can avoid manual cleanup.
  2. Instead of enivronment variables, use SHGetKnownFolderPath with FOLDERID_System to get %SystemRoot%\System32. This way we can avoid a dependency on environment variables.

Thanks for all your work on this.

Comment by Alessandro Gherardi [ 30/Dec/14 ]

Thanks, Mark. Without the MS hotfix and with tweaked code that never zeroes out data files, I could not reproduce the FlushViewOfFile error using dbtest, However, I could reproduce the error when creating a replicaset as mentioned in one of the posts in https://jira.mongodb.org/browse/SERVER-1459.

Here's another patch that uses file version info:
https://github.com/agherardi/mongo/compare/version
Please let me know how this looks.

Comment by Mark Benvenuto [ 29/Dec/14 ]

I was afraid of the relying on the file version checks since I did not know if they documented how they version releases. After reviewing a few links, the KBs are coming off the LDR (limited distribution release) branch. Any KB contains all the previous changes for that file.

I agree with your original suggestion. I think the best approach is now to check the file numbers by querying out VS_FIXEDFILEINFO from ntfs.sys.

In terms of reproing it, the original bug suggested

dbtest.exe --dur pdfile

You can build this executable if you run scons dbtest.exe.

Let me know if you need any help, or if you would like me to finish this.

Comment by Alessandro Gherardi [ 29/Dec/14 ]

Hi Mark,
I made the code changes you suggest and signed the contributor agreement.

Regarding the alternative approach - file versioning. I cannot install http://support.microsoft.com/kb/2731284 on my Windows 7 laptop - the update installer says "The update is not applicable to your computer". I'm wondering if the reason is that the version of ntfs.sys on my laptop is 6.1.7601.22297. That's in the 6.1.7601.22xxx range and is already higher than the version that the updater would like to install - 6.1.7601.22083 (see File Information in http://support.microsoft.com/kb/2731284).

If I disable zeroing out data files, is there an easy way to repro this issue https://jira.mongodb.org/browse/SERVER-1459?

Thanks,
Alessandro

Comment by Mark Benvenuto [ 29/Dec/14 ]

In terms of your tentative patch:

  1. Use COINIT_MULTITHREADED instead of COINIT_APARTMENTTHREADED
  2. In BSTRToString, call ConvertBSTRToString

    boost::scoped_array<char> temp = ConvertBSTRToString(...)
    return std::string(temp);

  3. Use _bstr_t instead of BSTR as much as possible.
  4. Use _com_ptr_t instead of having to call Release().
  5. See http://msdn.microsoft.com/en-us/library/5yb2sfxk.aspx for MSVC COM support

Your VARIANT usage is so simple that there is not much value for your ot switch to _variant_t.

To contribute this, you will need to sign: http://www.mongodb.com/legal/contributor-agreement. See Contributing for general information.

Thanks for your work on this.

Comment by Mark Benvenuto [ 29/Dec/14 ]

We already know which hotfix to check for so that is enough. I let the Windows installer ensure we get the right version of the files we need. I do not know if we can rely on higher version numbers of files containing all the the fixes of previous versions or that file numbers are linearly increasing with respect to the fixes they contain.

Comment by Alessandro Gherardi [ 27/Dec/14 ]

I have a question: Would checking the version of the file(s) updated by the hotfix be a better way to solve this problem? Specifically, for every file that the hotfix updates, we could check that the version of the file on disk is greater than or equal to the version contained in the hotfix. This approach is more reliable as often the same file is updated by multiple hotfixes.

Comment by Alessandro Gherardi [ 26/Dec/14 ]

See https://github.com/agherardi/mongo/compare/mongodb:master...master for a tentative patch.

Generated at Thu Feb 08 03:36:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.