[SERVER-38098] MongoDB's yaml-cpp is rejected by latest MSVC Created: 12/Nov/18  Updated: 29/Oct/23  Resolved: 15/Nov/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 3.4.22, 4.0.5

Type: Bug Priority: Major - P3
Reporter: Stephan T. Lavavej [X] Assignee: Henrik Edin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows


Backwards Compatibility: Fully Compatible
Sprint: Dev Tools 2018-11-19
Participants:

 Description   

The Microsoft Visual C++ team regularly builds MongoDB (and many other open source projects) so we can avoid introducing compiler and standard library bugs. Our build recently broke, due to MongoDB's use of yaml-cpp. Updating yaml-cpp to its latest master commit will fix the problem.

The full chain of events was:

  • yaml-cpp used a non-Standard macro `_NOEXCEPT` defined by the MSVC implementation.
  • MSVC removed the `_NOEXCEPT` macro in VS 2017 15.8 (released for production on Aug 14, 2018).
  • yaml-cpp fixed their usage of this macro on June 30, 2018: https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79
  • However, yaml-cpp 0.6.2 (predating this fix) is still the latest version.
  • Compiling MongoDB with VS 2017 15.8 or later emits compiler errors of the form "D:\MongoDB\src\src\third_party\yaml-cpp-0.6.2\include\yaml-cpp/exceptions.h(123): error C3646: '_NOEXCEPT': unknown override specifier".


 Comments   
Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-38098 Fix yaml-cpp compilation with Visual Studio 2017.

cherry picked from commit 0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79

https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79
(cherry picked from commit 866d4ba84171d7cc88876e2d0d8ae4eeb04ff019)
Branch: v3.4
https://github.com/mongodb/mongo/commit/9e73aceada701e0bf9fbd4730898ce21f68a9801

Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-38098 Fix yaml-cpp compilation with Visual Studio 2017.

cherry picked from commit 0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79

https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79
(cherry picked from commit 866d4ba84171d7cc88876e2d0d8ae4eeb04ff019)
Branch: v3.6
https://github.com/mongodb/mongo/commit/86723817c732feb06636251d3569199152492e31

Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-38098 Fix yaml-cpp compilation with Visual Studio 2017.

cherry picked from commit 0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79

https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79
(cherry picked from commit 866d4ba84171d7cc88876e2d0d8ae4eeb04ff019)
Branch: v4.0
https://github.com/mongodb/mongo/commit/12183cd80bbda93a9e94c803e182a649abaa360c

Comment by Stephan T. Lavavej [X] [ 15/Nov/18 ]

Great, thank you! mpark also fixed the questionable commit after I reported it.

Comment by Githook User [ 15/Nov/18 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-38098 Fix yaml-cpp compilation with Visual Studio 2017.

cherry picked from commit 0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79

https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79
Branch: master
https://github.com/mongodb/mongo/commit/866d4ba84171d7cc88876e2d0d8ae4eeb04ff019

Comment by Henrik Edin [ 15/Nov/18 ]

STL_MSFT I tried out the new VS 2017 15.9 compiler and managed to compile this with both /std:c+14 and /std:c+17. I'll let you know if we encounter any other constexpr bug but it seems to be good now. Also good catch on that linked commit, I never got around testing on other compilers.

Comment by Stephan T. Lavavej [X] [ 15/Nov/18 ]

Looking at the linked Developer Community issue, that was indeed a compiler regression (in the constexpr char_traits compiler builtins; I worked on the library side of that). It was marked as fixed in VS 2019 16.0, the upcoming major version (which will be binary compatible with the 2015 and 2017 release series). However, it was not marked as ported to the VS 2017 15.9 release that just became available. The constexpr char_traits compiler builtins are used in C+17 mode only; if you're compiling with /std:c14 (the default) then that bug should not affect you. The DevComm thread mentioned that there was a second constexpr bug, which is entirely plausible, and would explain how /std:c+14 mode could also be affected. I am uncertain as to whether any other bugs were fixed here.

  1. I am confused by the linked commit, as it appears to be enabling MPARK_CPP14_CONSTEXPR when the feature-test macro is sufficiently new (this is now defined by MSVC) and the compiler reports itself as MSVC, but other than 19.15. That seems unintentional (e.g. GCC and Clang on Linux won't activate this because they don't define `_MSC_VER`). I would expect to see "enable MPARK_CPP14_CONSTEXPR if the feature-test macro is sufficiently new, and either the compiler isn't MSVC, or it is MSVC other than 19.15".
  2. If you can provide a self-contained test case, including only mpark/variant with the MSVC workaround commented out, that misbehaves with VS 2017 15.9's constexpr implementation in C++14 mode (where the constexpr char_traits issue can't be a problem), I can send it to the compiler team.
Comment by Henrik Edin [ 14/Nov/18 ]

STL_MSFT Thank you for reporting this, I should have it committed shortly

When I have you on a thread. I discovered an issue with the latest version of the compiler in another third party library we use. They have committed a "fix" to disable constexpr and they make it sound like a regression in the compiler. Do you have any insight on this whether their implementation is incorrect or if there's a regression with constexpr?

The commit in question: https://github.com/mpark/variant/commit/1a861281fc1c6fd2037d050a70ff53dcbb635d93
Issue thread: https://github.com/mpark/variant/issues/48

Apparently it worked with previous 19.x versions of cl.exe. We need this because we will compile with /std:c++14 as an intermediate step.

Comment by Mark Benvenuto [ 13/Nov/18 ]

henrik.edin as discussed, I am assigning you this ticket since you are working on migrating MongoDB to VS 2017.8

Comment by Stephan T. Lavavej [X] [ 13/Nov/18 ]

Yep, that should work. (The author of that commit, alexkaratarakis, is an MSVC dev like me.) Please let me know if you encounter any compiler/library issues while porting to 2017 15.8; I'm a C++ Standard Library maintainer and if I can't solve your issues, I can put you in contact with the compiler devs who can.

Comment by Mark Benvenuto [ 12/Nov/18 ]

We are in-progress of switching to VS 2017.8. We have not tried in a few months. It looks like we will have to patch our version of yaml-cpp with what may be the fix: https://github.com/jbeder/yaml-cpp/commit/0f9a586ca1dc29c2ecb8dd715a315b93e3f40f79

Comment by Stephan T. Lavavej [X] [ 12/Nov/18 ]

We're building https://github.com/mongodb/mongo/commit/efea6a2c614627152ada31468462033ec0f7ffed . Please note that yaml-cpp 0.6.2 is affected by the bug - yaml-cpp master has been fixed.

(Given how our build recently broke, I suspect that MongoDB updating yaml-cpp to 0.6.2 was the triggering factor, but I haven't done any source code archaeology to confirm.)

Do you regularly build with VS 2017's latest update (currently 15.8, soon 15.9)? This issue should always reproduce with those versions.

Comment by Mark Benvenuto [ 12/Nov/18 ]

STL_MSFT Which version of MongoDB are you building? We recently updated to yaml-cpp-0.6.2 in master as part of SERVER-37984. We will not be porting this change to our stable released versions like 4.0 of MongoDB though.

Comment by A. Jesse Jiryu Davis [ 12/Nov/18 ]

Thanks. This is part of the MongoDB server project, not the MongoDB C++ Driver, so I moved it from the CXX project in Jira to the SERVER project.

Generated at Thu Feb 08 04:47:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.