[SERVER-59482] Illegal instruction on Debian 11 Created: 20/Aug/21  Updated: 27/Oct/23  Resolved: 24/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.2
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Koichi MATSUMOTO Assignee: Andrew Morrow (Inactive)
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OS: Debian 11.0 (bullseye)

gcc (Debian 10.2.1-6) 10.2.1 20210110
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 

Python 3.9.6


Issue Links:
Related
related to DOCS-14757 Document the need to disable hyper-v ... Closed
is related to SERVER-58648 Coredump when running version 5.0.0 m... Closed
Operating System: ALL
Steps To Reproduce:
  1. git clone https://github.com/mongodb/mongo.git
  2.  pip install -r etc/pip/compile-requirements.txt
  3. ./buildscripts/scons.py install-core --disable-warnings-as-error
Sprint: Dev Platform 2021-08-23, Dev Platform 2021-09-06
Participants:

 Description   

I tried to build mongod on master branch, but mongod crashed at starting.

Mongod only prints Illegal instruction.

  • gdb result

 

$ ./mongod --version
Illegal instruction
$ gdb mongod
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from mongod...
(gdb) b main
Breakpoint 1 at 0x1256d20: main. (6 locations)
(gdb) r --version
Starting program: /home/mzch/mongo/build/install/bin/mongod --version
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGILL, Illegal instruction.
tcmalloc::SizeMap::Init (this=this@entry=0x55555a8087a0 <tcmalloc::Static::sizemap_>)
    at src/third_party/gperftools/dist/src/common.cc:319319
     num_objects_to_move_[cl] = NumMoveSize(ByteSizeForClass(cl));
(gdb) p cl
$1 = 1
(gdb) s
Program received signal SIGILL, Illegal instruction.
tcmalloc::SizeMap::NumMoveSize (size=<optimized out>, this=<optimized out>)
    at src/third_party/gperftools/dist/src/common.cc:181181
 int SizeMap::NumMoveSize(size_t size) {
(gdb) p size
$2 = <optimized out>
(gdb) s
Program received signal SIGILL, Illegal instruction.
0x00005555592318dc in tcmalloc::SizeMap::NumMoveSize (this=<optimized out>, size=<optimized out>)
    at src/third_party/gperftools/dist/src/common.cc:197197
   if (num > FLAGS_tcmalloc_transfer_num_objects)
(gdb) p num
$3 = <optimized out>
(gdb) p FLAGS_tcmalloc_transfer_num_objects
$4 = 32
(gdb) s
Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
(gdb)

Could anyone tell me what’s wrong?

 



 Comments   
Comment by Andrew Morrow (Inactive) [ 24/Aug/21 ]

koichi@matsumoto.me - Glad to hear that worked for you. I'm going to close out this ticket.

Comment by Koichi MATSUMOTO [ 24/Aug/21 ]

I built mongodb with `CCFLAGS=-march=westmere` then mongod started successfully.

Thanks!

Comment by Andrew Morrow (Inactive) [ 23/Aug/21 ]

zone@phantasea.net - Thanks for trying that out, and I'm glad it resolved your issue. Thanks as well for confirming back to us - we will make some documentation changes to highlight the need to disable hyper-v when using VirtualBox on Windows.

Comment by Dustin Brockert [ 23/Aug/21 ]

As soon as I turned off my Hyper-V service (Hyper-V Host Compute Service) then the mongo and mongod functioned as expected. Thanks!

Comment by Andrew Morrow (Inactive) [ 23/Aug/21 ]

As of v5.0, prebuilt MongoDB binaries require a processor that supports AVX instructions (for more details, see https://docs.mongodb.com/manual/administration/production-notes/#x86_64).

If you are building from source, you can downgrade this requirement by adding CCFLAGS=-march=<target> to your SCons invocation, where <target> is your desired target microarchitecture as defined by https://gcc.gnu.org/onlinedocs/gcc-8.5.0/gcc/x86-Options.html#x86-Options.

If you are sticking with the prebuilt v5.0 binaries, it is important to ensure that both your physical hardware offers AVX instructions, and that any virtualization solution that is in play exposes that support for AVX to the guest.

  • koichi@matsumoto.me, if the hardware on which you would like to run doesn't support AVX, I'd suggest rebuilding from source with a different value for the -march flag and you should no longer observe the illegal instruction. Alternatively, run the software on a system which supports AVX instructions.
  • zone@phantasea.net, a quick look around for details on that Ryzen processor suggests that it should support AVX, so I'd look to your VirtualBox setup to see why the guest OS doesn't seem to have access. If your host OS is Windows, we have seen some indications that disabling hyper-v may be required, per https://forums.virtualbox.org/viewtopic.php?f=6&t=101917. If you could see if that helps for you and report back it would be valuable feedback.
Comment by Dustin Brockert [ 21/Aug/21 ]

Running into the same issue with Ubuntu 20.04 with both the premade binaries and compiling like above. It is running under a virtual machine in VirtualBox for reference. Also works with the latest 4.4 release.

CPU Info:
processor : 0
processor :0
vendor_id : AuthenticAMD
cpu family : 25
model : 33
model name : AMD Ryzen 9 5900X 12-Core Processor
stepping : 0
microcode : 0xffffffff
cpu MHz : 3700.006
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx fxsr_opt rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid tsc_known_freq pni ssse3 sse4_1 sse4_2 hypervisor lahf_lm cmp_legacy cr8_legacy 3dnowprefetch vmmcall fsgsbase arat
bugs : fxsave_leak sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass
bogomips : 7400.01
TLB size : 2560 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management:

Generated at Thu Feb 08 05:47:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.