[SERVER-35935] unittest/unittest_test StackTraceForAssertion failed Created: 02/Jul/18 Updated: 27/Oct/23 Resolved: 30/Jul/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | 4.0.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Marek Skalický | Assignee: | Andrew Morrow (Inactive) |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
GCC 8.1.1 compiler, glibc 2.27.9000 |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Steps To Reproduce: | $ cat build-options $ scons $(cat build-options) build/fedora/mongo/unittest/unittest_test --opt=off |
||||||||
| Participants: | |||||||||
| Description |
|
unittest/unittest_test fails in MongoDB 4.0 - output is attached. Or is something special needed to be able to print stacktrace properly? |
| Comments |
| Comment by Marek Skalický [ 30/Jul/18 ] | ||||||
|
Sure, you can close this issue. If I could somehow help with Fedora compile flags, please feel free to contact me. | ||||||
| Comment by Andrew Morrow (Inactive) [ 18/Jul/18 ] | ||||||
|
Thanks mskalick - Some of those flags do look interesting. I've filed | ||||||
| Comment by Marek Skalický [ 12/Jul/18 ] | ||||||
|
Thanks both of you for deep analysis. Removing "-fvisibility=hidden" fixed the unit test failure. I'm sorry, but I don't know much about mentioned gcc flags - as it was noted, they are distribution default. They are described in https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc/ . I don't know about performance impact so I asked in this thread, so I hope some gcc person will reply. To note: I'm going to have two week PTO. I'll return to this after it. | ||||||
| Comment by Mark Benvenuto [ 12/Jul/18 ] | ||||||
|
Here is Fedora's discussion about their default options: | ||||||
| Comment by Mark Benvenuto [ 09/Jul/18 ] | ||||||
|
There are only 3 flags that are added in the spec file for MongoDB. The rest are system-wide RPM build defaults for Fedora. https://src.fedoraproject.org/rpms/mongodb/blob/master/f/mongodb.spec#_233-234
| ||||||
| Comment by Andrew Morrow (Inactive) [ 09/Jul/18 ] | ||||||
|
mskalick - Looking at that list of flags, there are several things that I think you either shouldn't do, or don't need to do, when building 4.0:
I don't know what is in those various hardening scripts, so hard to say what they do. I am somewhat interested in whether you think we should be applying these flags for our hardened builds. Do you have some guidance on the runtime costs?
I'm also curious about why you are applying -fexceptions and -fasynchronous-unwind-tables. The first seems unnecessary for a C++ project, and the second seems potentially interesting, if you have some context on why it is applied. | ||||||
| Comment by Mark Benvenuto [ 05/Jul/18 ] | ||||||
|
mskalick The problem is the "-fvisibility=hidden" flag which was introduced in this commit. If you remove this flag, the unittest will pass. Removing this flag will be beneficial since MongoDB automatically prints stack traces which allows us to often quickly troubleshoot many issues before loading crash dumps in the debugger. | ||||||
| Comment by Mark Benvenuto [ 03/Jul/18 ] | ||||||
|
mskalick Using Koji, I grabbed a 4.0 build, and examined how it was compiling. Here is the variables I found in the Koji logs:
I found that both MongoDB 4.0 on Fedora 28 with gcc-8.1.1-1.fc28 and binutils-2.29.1-23.fc28 and the official mongodb-3.6.3-1.fc28 package both fail to print mongod function names when I send SIGTERM to mongod. This means that one or more of these flags is causing dladdr to not find the function name. It will take me some time to isolate the problematic flag. | ||||||
| Comment by Andrew Morrow (Inactive) [ 03/Jul/18 ] | ||||||
|
mskalick - We don't do anything particular to achieve that, usually. We hard code that we link with -rdynamic in the top level Sconstruct (look for env.Append(LINKFLAGS=['-rdynamic'])) to ensure that the backtracer is able to find symbols. I'm not aware of any other options. If you downgrade to GCC 7, does this test pass on that same system? | ||||||
| Comment by Marek Skalický [ 03/Jul/18 ] | ||||||
And what compiler or whatever options are needed to get resolvable names of symbols? | ||||||
| Comment by Mark Benvenuto [ 02/Jul/18 ] | ||||||
|
The exception message is:
The exception happens because the backtrace code cannot resolve the names of almost any symbols in the unittest binary. |