[SERVER-13824] V8-3.12 Segfaults when compiled with GCC 4.9.0 Created: 02/May/14  Updated: 13/Jul/15  Resolved: 21/Jul/14

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mathias Stearn Assignee: Adam Midvidy
Resolution: Won't Fix Votes: 0
Labels: community-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-13809 switch default JavaScript engine from... Closed
Duplicate
is duplicated by SERVER-16985 Segmentation fault in mongo, V8 Closed
is duplicated by SERVER-17238 $group with invalid reduce step crash... Closed
is duplicated by SERVER-17239 segfault running group using CodeWSco... Closed
is duplicated by SERVER-17593 Segfault in V8 when running Java driv... Closed
is duplicated by SERVER-13880 mongo client from ArchLinux packages ... Closed
Related
is related to SERVER-17593 Segfault in V8 when running Java driv... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Server 2.7.3, Server 2.7.4
Participants:

 Description   

2014-05-02T17:29:06.043-0400 mongo got signal 11 (Segmentation fault), stack trace:
2014-05-02T17:29:06.048-0400 0x7d8e76 0x61fca9 0x7f40a6043df0 0xa44b9c 0xab1ff8 0xab2034 0xab2cd9 0xab6f1b 0xab73fa 0xa4b49e 0xa5b0ed 0xa50988 0xa5b7c6 0xa50a10 0xa5b7c6 0xa50a10 0xa5b7c6 0xa50a10 0xa5b7c6 0xa50a10
 /home/mstearn/10gen/mongo/mongo(_ZN5mongo15printStackTraceERSo+0x26) [0x7d8e76]
 /home/mstearn/10gen/mongo/mongo(_Z12quitAbruptlyi+0x109) [0x61fca9]
 /usr/lib/libc.so.6(+0x33df0) [0x7f40a6043df0]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6String11WriteToFlatItEEvPS1_PT_ii+0x3bc) [0xa44b9c]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal33GenericStringUtf16CharacterStream10FillBufferEjj+0x38) [0xab1ff8]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal28BufferedUtf16CharacterStream9ReadBlockEv+0x34) [0xab2034]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal7Scanner20SkipMultiLineCommentEv+0x129) [0xab2cd9]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal7Scanner4ScanEv+0x10eb) [0xab6f1b]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal7Scanner4NextEv+0x4a) [0xab73fa]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser6ExpectENS0_5Token5ValueEPb+0x5e) [0xa4b49e]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser10ParseBlockEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x31d) [0xa5b0ed]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser14ParseStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x98) [0xa50988]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser16ParseIfStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser14ParseStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser16ParseIfStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser14ParseStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser16ParseIfStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser14ParseStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser16ParseIfStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(_ZN2v88internal6Parser14ParseStatementEPNS0_8ZoneListINS0_6HandleINS0_6StringEEEEEPb+0x120) [0xa50a10]



 Comments   
Comment by Githook User [ 31/Jul/14 ]

Author:

{u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-13824: Remove some typing violations from the v8 code

Code will now work correctly with GCC 4.9.x with full optimization.
Branch: master
https://github.com/mongodb/mongo/commit/0028a339876b352575758910a158973852ce4cfe

Comment by Geert Bosch [ 26/Jul/14 ]

I tried gcc (GCC) 4.9.2 20140725 (prerelease) and it failed in similar fashion. As GCC developer I wanted to extract a bug report for GCC (if it was indeed a bug in GCC) or otherwise find out what was wrong. A look at the sources in v8utils.h clearly indicates the problem. The assumption is that it is OK to use reinterpret_cast to convert between pointer types. This is undefined in general, but entirely unreasonable if the types have different alignment as is the case here. Indeed the whole purpose of the code is to ignore alignment. True, you might get away with it in some cases, but that doesn't make it correct.

Note that the whole notion of "V8_HOST_CAN_READ_UNALIGNED" is flawed. Can read what unaligned? An integer? A floating point double? A 32-byte vector? Answers can differ, and the V8 code is utterly broken in this regard. Unfortunately, even if V8_HOST_CAN_READ_UNALIGNED is false, there is a reinterpret_cast of pointers to uint32_t (even on 64-bit systems!) to try and manually optimize loops for which GCC/glibc would use optimal code if memcmp had simply been called instead of CompareRawStringContents.

I have a patch that includes a minimal fix to v8 that essentially omits setting V8_HOST_CAN_READ_UNALIGNED and fixes CompareRawStringContents by removing the code that tries to compare by words.

-Geert

Comment by Matt Kangas [ 21/Jul/14 ]

GCC 4.9.1 has been released and appears to resolve this issue. We are no longer able to reproduce the segfault.

If you must compile with GCC 4.9.0, recommended workarounds include:

  • Compile with -O2 rather than -O3
  • Compile with -fno-tree-vectorize

Note: our SConstruct does not provide a way to specify alternate compiler options, so you have to hack this line to specify "-O2" rather than "-O3".
https://github.com/mongodb/mongo/blame/14ec31c/SConstruct#L908

Comment by Adam Midvidy [ 16/Jul/14 ]

Further investigation shows that this is probably due to an unaligned access occurring in vectorized code,

Here is the code generated with -O2 (no crash occurs)

      *reinterpret_cast<uintptr_t*>(dest) =
  a3c398:	48 8b 0a             	mov    (%rdx),%rcx
          *reinterpret_cast<const uintptr_t*>(src);
      dest += kStepSize;
  a3c39b:	48 83 c0 08          	add    $0x8,%rax
      src += kStepSize;
  a3c39f:	48 83 c2 08          	add    $0x8,%rdx
      return;
    }
    // Number of characters in a uintptr_t.
    static const int kStepSize = sizeof(uintptr_t) / sizeof(*dest);  // NOLINT
    while (dest <= limit - kStepSize) { }}

Here is the same code compiled with -O3 (the crash occurs at instruction 0xa66b7c)

   // Number of characters in a uintptr_t.
    static const int kStepSize = sizeof(uintptr_t) / sizeof(*dest);  // NOLINT
    while (dest <= limit - kStepSize) {
      *reinterpret_cast<uintptr_t*>(dest) =
  a66b7c:	66 41 0f 6f 44 15 00 	movdqa 0x0(%r13,%rdx,1),%xmm0
  a66b83:	49 83 c0 01          	add    $0x1,%r8
  a66b87:	41 0f 11 04 12       	movups %xmm0,(%r10,%rdx,1)
  a66b8c:	48 83 c2 10          	add    $0x10,%rdx
  a66b90:	4d 39 c8             	cmp    %r9,%r8
}

Note that the loop has been vectorized to use the xmm registers using the movdqa instruction. From the intel arch documentation: "When the source or destination operand is a memory operand, the operand must be aligned on a 16-byte boundary or a general-protection exception (#GP) will be generated." As there is no requirement that the input to CopyChars be aligned, the use of the xmm registers triggers the fault.

Furthermore, clang with -O3 does not emit vector instructions for the function in question.

Comment by Matt Kangas [ 15/May/14 ]

In SERVER-13880 a user reported that ArchLinux's build of 2.6.1-1 (using gcc 4.9) produced a similar failure:

Just type "db.abc.ensureIndex(
{'m':1}
)" in mongo-shell.

We need do a full test suite run against v2.6 built with gcc-4.9 and see what falls out.

Comment by Matt Kangas [ 15/May/14 ]

Demangled stack trace:

 /home/mstearn/10gen/mongo/mongo(mongo::printStackTrace(std::basic_ostream<char, std::char_traits<char> >&)+0x26) [0x7d8e76]
 /home/mstearn/10gen/mongo/mongo(quitAbruptly(int)+0x109) [0x61fca9]
 /usr/lib/libc.so.6(+0x33df0) [0x7f40a6043df0]
 /home/mstearn/10gen/mongo/mongo(void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int)+0x3bc) [0xa44b9c]
 /home/mstearn/10gen/mongo/mongo(v8::internal::GenericStringUtf16CharacterStream::FillBuffer(unsigned int, unsigned int)+0x38) [0xab1ff8]
 /home/mstearn/10gen/mongo/mongo(v8::internal::BufferedUtf16CharacterStream::ReadBlock()+0x34) [0xab2034]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Scanner::SkipMultiLineComment()+0x129) [0xab2cd9]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Scanner::Scan()+0x10eb) [0xab6f1b]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Scanner::Next()+0x4a) [0xab73fa]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::Expect(v8::internal::Token::Value, bool*)+0x5e) [0xa4b49e]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseBlock(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x31d) [0xa5b0ed]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x98) [0xa50988]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseIfStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseIfStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseIfStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x120) [0xa50a10]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseIfStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x1d6) [0xa5b7c6]
 /home/mstearn/10gen/mongo/mongo(v8::internal::Parser::ParseStatement(v8::internal::ZoneList<v8::internal::Handle<v8::internal::String> >*, bool*)+0x120) [0xa50a10]

Generated at Thu Feb 08 03:33:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.