[SERVER-25689] Add Memory Sanitizer builds to Evergreen Created: 18/Aug/16  Updated: 02/Feb/24

Status: Backlog
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File msan.tar.gz    
Issue Links:
Related
related to SERVER-20588 Enable MemorySanitizer Backlog
related to CDRIVER-3317 Replace valgrind variant with Memory ... Closed
Assigned Teams:
Build
Participants:

 Description   

Memory Sanitizer sort of works on Ubuntu 16.04.

At this point, we have three options for how we want to use it:
1. Community Build
2. Community + SSL Build
3. Enterprise Build

While #3 gives us the most coverage, it requires us to recompile all the third-party libraries we use (SSL, Cyrus SASL, Kerberos, Ldap, Curl, etc), and I do not think OpenSSL will work.

At the minimum, we need to build our own version of libc++, and install it on the builders we use for compiling and testing.

Background
In order to use it, you need to recompile all third-party libraries with it. This means the C++ standard library, and third party libraries like OpenSSL, and Cyrus SASL. You do not need to recompile GLIBC. In my testing, I have only recompiled libC++, and OpenSSL.

Do you really need to recompile all the libraries? Yes, if you do not recompile the libraries for an enterprise build, you get a lot of false positives. Some of these show up in our code:

SUMMARY: MemorySanitizer: use-of-uninitialized-value /mnt/d/mark/mongo/src/mongo/crypto/crypto_test.cpp:55:9 in mongo::(anonymous namespace)::UnitTest__CryptoVectors__SHA1::_doTest()
SUMMARY: MemorySanitizer: use-of-uninitialized-value /mnt/d/mark/mongo/src/mongo/db/modules/enterprise/src/encryptdb/symmetric_crypto_smoke.cpp:165:45 in mongo::crypto::smokeTestAESCipherMode(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/src/debug/cyrus-sasl-2.1.26/lib/config.c:75 in sasl_config_init
SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/src/debug/cyrus-sasl-2.1.26/lib/dlopen.c:279 in _parse_la
SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/src/debug/cyrus-sasl-2.1.26/lib/dlopen.c:485 in _sasl_load_plugins
SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/src/debug/openssl-1.0.2h/crypto/rand/md_rand.c:224 in ssleay_rand_add

Now if you do recompile say openssl with memory sanitizer, you hit this instead,

#0  0x00007ffff6434075 in OPENSSL_cpuid_setup () from /data/msan/lib/libcrypto.so.1.0.0
#1  0x00007ffff64283b2 in _init () from /data/msan/lib/libcrypto.so.1.0.0
#2  0x0000000000000001 in ?? ()
#3  0x00007ffff7de74aa in call_init (l=0x7ffff7fe59c8, argc=argc@entry=1, argv=argv@entry=0x7fffffffebb8, env=env@entry=0x7fffffffebc8) at dl-init.c:58
#4  0x00007ffff7de75fb in call_init (env=0x7fffffffebc8, argv=0x7fffffffebb8, argc=1, l=<optimized out>) at dl-init.c:30
#5  _dl_init (main_map=0x7ffff7ffe168, argc=1, argv=0x7fffffffebb8, env=0x7fffffffebc8) at dl-init.c:120
#6  0x00007ffff7dd7cfa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#7  0x0000000000000001 in ?? ()
#8  0x00007fffffffedc6 in ?? ()
#9  0x0000000000000000 in ?? ()

Chromium and LLVM are big users of Memory Sanitizer. Chromium uses NSS for SSL support, not OpenSSL.



 Comments   
Comment by Eric Milkie [ 18/Aug/16 ]

OpenSSL used to have a build flag to make it work without false positives when used with Valgrind Memcheck. Perhaps that could work for Memory Sanitizer as well?

Comment by Mark Benvenuto [ 18/Aug/16 ]

Here are some sample files to to build MSAN enabled libc++, and openssl enabled msan.

Generated at Thu Feb 08 04:09:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.