[SERVER-12991] Segmentation fault during V8 initialization on grsecurity Linux kernel Created: 03/Mar/14 Updated: 11/Aug/15 Resolved: 28/Mar/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 2.6.0-rc0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Miroslav Zacek | Assignee: | Jonathan Reams |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Steps To Reproduce: | 1. Use kernel GR patched kernel: http://molly.corsac.net/~corsac/debian/kernel-grsec/packages/wheezy/
run
and reboot 2. Install mongodb-org from http://downloads-distro.mongodb.org:
to apt sources
3. run
|
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
On GRSEC + PAX patched kernel the mongo client crashes (segmentation fault)
I've experienced the same error with the 2.5.5 version. I've installed the unstable version because previous versions failed on accessing /sys/devices which is banned by GRSEC patch. Server is now able to start but the client crashes. system: Debian wheezy
The tail of the strace log around the crash:
|
| Comments |
| Comment by Matt Kangas [ 28/Mar/14 ] | |||||||||||||||||||||||||||||||
|
V8, the Javascript engine used by mongodb, requires the ability to write executable pages of memory for Just-In-Time (JIT) compilation. If an operating system has been configured so it's not possible to write to executable memory regions, V8 cannot function. The impact to MongoDB in this case is:
It is possible that mongod may start successfully but will fail as soon as the scripting engine is used. For Linux grsecurity patched kernels, the recommended workaround is to set the following PaX flags.
| |||||||||||||||||||||||||||||||
| Comment by Jonathan Reams [ 28/Mar/14 ] | |||||||||||||||||||||||||||||||
|
Also see the application-specific settings for grsecurity: On the section for google chrome - which also uses V8, you have to turn off the mprotect patches for the chrome to get it to work properly. | |||||||||||||||||||||||||||||||
| Comment by Jonathan Reams [ 23/Mar/14 ] | |||||||||||||||||||||||||||||||
|
As an experiment, I patched the LOG macro so that logging would be initialized if it wasn't already - thus getting past this particular failure. That just moved the segfault down the road a bit:
| |||||||||||||||||||||||||||||||
| Comment by Jonathan Reams [ 23/Mar/14 ] | |||||||||||||||||||||||||||||||
|
This was easily reproducible with the provided vagrant box. The segfault happens because logging isn't initialized when LOG is called after the allocation failure. I was able to get mongo to start relatively successfully by setting the correct PaX flags that disable some of the grsec protections:
| |||||||||||||||||||||||||||||||
| Comment by Miroslav Zacek [ 17/Mar/14 ] | |||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||
| Comment by Matt Kangas [ 10/Mar/14 ] | |||||||||||||||||||||||||||||||
|
According to the addr2line output, the segfault occurred during V8 initialization. Specifically:
The mmap() does not succeed on your system. While attempting to log the message "mmap failed", a segfault occurs. I suspect it's because i::Isolate::Current() returns NULL. If it wasn't for the LOG failure, the rest of this code seems like it should handle the mmap() failure with fallback transcendental functions. In any event, the steps to resolving this are:
Can you point us to a Vagrant configuration (or something similar) which makes it easy to reproduce this failure? | |||||||||||||||||||||||||||||||
| Comment by Matt Kangas [ 04/Mar/14 ] | |||||||||||||||||||||||||||||||
|
addr2line output from release artifacts with debug symbols.
| |||||||||||||||||||||||||||||||
| Comment by Matt Kangas [ 03/Mar/14 ] | |||||||||||||||||||||||||||||||
|
Hmm, is it really gperftools? That's the only other reference to /sys/devices that we haven't explicitly guarded.
| |||||||||||||||||||||||||||||||
| Comment by Matt Kangas [ 03/Mar/14 ] | |||||||||||||||||||||||||||||||
|
Note, this did not occur on a stock Linux kernel for any distro, but a specific security-patched kernel (grsecurity) Related: | |||||||||||||||||||||||||||||||
| Comment by Matt Kangas [ 03/Mar/14 ] | |||||||||||||||||||||||||||||||
|
Demangled stack trace:
|