[SERVER-58648] Coredump when running version 5.0.0 mongo/mongod Created: 17/Jul/21  Updated: 12/Jan/22  Resolved: 09/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ali Molaei Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-59482 Illegal instruction on Debian 11 Closed
related to DOCS-14684 Clarify requirement for "Sandy Bridge... Closed
Operating System: ALL
Steps To Reproduce:

Run commands:

 

mongo
mongod

 

Participants:
Case:

 Description   

I'm maintaining mongodb-bin on ArchLinux's AUR, running mongo or mongod command, I'm getting this systemd-coredump:

Jul 17 13:05:51 B590 systemd-coredump[6628]: [🡕] Process 6626 (mongo) of user 1000 dumped core.
 
 Found module linux-vdso.so.1 with build-id: 581bfd6488519395b6a2f6b6d72da5daf689fa5d
 Found module libkeyutils.so.1 with build-id: ac405ddd17be10ce538da3211415ee50c8f8df79
 Found module libkrb5support.so.0 with build-id: c4ee4ad1dc2da57487bc2419b88f1b6873184582
 Found module libcom_err.so.2 with build-id: 86413f38be27bd051645eb9fb0ad0dcace5b2872
 Found module libk5crypto.so.3 with build-id: 632a59ed7c35704d84645e6d1e9873348d1eb802
 Found module libkrb5.so.3 with build-id: c61cb4da63b8a839ee7df99eaf9dbd3d0968534c
 Found module libunistring.so.2 with build-id: 015ac6d6bcb60b7d8bea31a80d1941b06e8636ab
 Found module libz.so.1 with build-id: 81bf6e728a6d6f5b105b0f8b25f6c614ce10452a
 Found module libzstd.so.1 with build-id: 4b10444c1560ebc574af4d5f488b7408b22d450e
 Found module libgssapi_krb5.so.2 with build-id: 9be9d3348399b72b76161a64e6d9fd760b77163a
 Found module libpsl.so.5 with build-id: 0229a201aaf5652186c9fdc192ebe52baf19d7f1
 Found module libssh2.so.1 with build-id: 7f6d9edd2e793b266cae4f22fc1ba7b6b401c08c
 Found module libidn2.so.0 with build-id: f0fd7579e163a1ff289a101d0bce7659de9581a0
 Found module libnghttp2.so.14 with build-id: 9d82c05f5a6f78e3c72b3dea2180f2e209fc916c
 Found module libgcc_s.so.1 with build-id: 7f8508bb914546ada778809b64b99d234337d835
 Found module ld-linux-x86-64.so.2 with build-id: 040cc3dd10461562f177df39e3be2f3704258c3c
 Found module libc.so.6 with build-id: 4b406737057708c0e4c642345a703c47a61c73dc
 Found module libm.so.6 with build-id: 2b8fd1f869ecab4e0b55e92f2f151897f6818acf
 Found module libpthread.so.0 with build-id: 07c8f95b4f3251d08550217ad8a1f31066229996
 Found module librt.so.1 with build-id: 75484da2d6f1515189eefa076e0a40328834cd16
 Found module libdl.so.2 with build-id: 5abc547e7b0949f89f3c0e21ab0c8331a7440a8a
 Found module libssl.so.1.1 with build-id: 99394a6653d9c107f2e9b730bbbfd18ed43ae3b9
 Found module libcrypto.so.1.1 with build-id: 81b77a8e0b6e1c0db19644a5f120890f02762021
 Found module libresolv.so.2 with build-id: c915c72668282861a813f7ea3c0780f37b681dc0
 Found module liblzma.so.5 with build-id: f8d607153b2b315f1aaa1c8e12553d544e661f2d
 Found module libcurl.so.4 with build-id: c4cd766f60888330b65feaf5a8b42b84e225be53
 Found module mongo with build-id: ff433d52b10b258fa6c0223d809d8f133b350416
 Stack trace of thread 6626:
 #0 0x0000562014a2a0da _ZN8tcmalloc7SizeMap4InitEv (mongo + 0x1feb0da)
 #1 0x0000562014a329f7 _ZN8tcmalloc6Static14InitStaticVarsEv (mongo + 0x1ff39f7)
 #2 0x0000562014a34447 _ZN8tcmalloc11ThreadCache10InitModuleEv (mongo + 0x1ff5447)
 #3 0x0000562014a345dd _ZN8tcmalloc11ThreadCache22CreateCacheIfNecessaryEv (mongo + 0x1ff55dd)
 #4 0x0000562014adc995 _ZN8tcmalloc27allocate_full_cpp_throw_oomEm (mongo + 0x209d995)
 #5 0x000056201350b97b _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St2>
 #6 0x00005620134a11f6 n/a (mongo + 0xa621f6)
 #7 0x0000562014adb24d __libc_csu_init (mongo + 0x209c24d)
 #8 0x00007f8560d29ab1 __libc_start_main (libc.so.6 + 0x27ab1)
 #9 0x00005620134fed2e _start (mongo + 0xabfd2e)
░░ Subject: Process 6626 (mongo) dumped core
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ Documentation: man:core(5)
░░ 
░░ Process 6626 (mongo) crashed and dumped core.
░░ 
░░ This usually indicates a programming error in the crashing program and
░░ should be reported to its vendor as a bug

I don't know how can I provide more information, glad to help if I can.

 

BTW, These are files I got from mongodb-org official ubuntu repo:

https://repo.mongodb.org/apt/ubuntu/dists/focal/mongodb-org/5.0/multiverse/binary-amd64/mongodb-org-shell_5.0.0_amd64.deb

https://repo.mongodb.org/apt/ubuntu/dists/focal/mongodb-org/5.0/multiverse/binary-amd64/mongodb-org-server_5.0.0_amd64.deb

https://repo.mongodb.org/apt/ubuntu/dists/focal/mongodb-org/5.0/multiverse/binary-amd64/mongodb-org-mongos_5.0.0_amd64.deb

mongodb-bin /etc/
mongodb-bin /etc/mongodb.conf
mongodb-bin /usr/
mongodb-bin /usr/bin/
mongodb-bin /usr/bin/mongo
mongodb-bin /usr/bin/mongod
mongodb-bin /usr/bin/mongos
mongodb-bin /usr/lib/
mongodb-bin /usr/lib/systemd/
mongodb-bin /usr/lib/systemd/system/
mongodb-bin /usr/lib/systemd/system/mongodb.service
mongodb-bin /usr/lib/sysusers.d/
mongodb-bin /usr/lib/sysusers.d/mongodb.conf
mongodb-bin /usr/lib/tmpfiles.d/
mongodb-bin /usr/lib/tmpfiles.d/mongodb.conf
mongodb-bin /usr/share/
mongodb-bin /usr/share/licenses/
mongodb-bin /usr/share/licenses/mongodb-bin/
mongodb-bin /usr/share/licenses/mongodb-bin/LICENSE
mongodb-bin /usr/share/man/
mongodb-bin /usr/share/man/man1/
mongodb-bin /usr/share/man/man1/mongo.1.gz
mongodb-bin /usr/share/man/man1/mongod.1.gz
mongodb-bin /usr/share/man/man1/mongos.1.gz

 



 Comments   
Comment by Eric Sedor [ 28/Jul/21 ]

Thanks for this report ali.molaei@protonmail.com; based on it we've opened DOCS-14684 to clarify our production notes.

Comment by Andrew Morrow (Inactive) [ 28/Jul/21 ]

ali.molaei@protonmail.com - Thanks for the update. Given that, I think it is conclusive that AVX support is necessary, but it is important also to note that AVX alone is not sufficient. The GCC 8 manual states that -march=sandybridge enables codegen assuming the following:

Intel Sandy Bridge CPU with 64-bit extensions, MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AES and PCLMUL instruction set support.

So you need a CPU that offers at least all of those features, and the Celeron doesn't. I don't actually know if we generate any POPCNT, AES, or PCLMUL instructions, but there is nothing stopping the compiler from doing so if it chooses.

Comment by Ali Molaei [ 28/Jul/21 ]

Oh my bad, I miss checked the flags, correct table is:

Celeron | i5 | i7

--------------------------

NULL | avx       | avx

NULL | f16c      | f16c

NULL | rdrand  | rdrand

NULL | ida        | ida

NULL | NULL     | aes

NULL | NULL     | smx

Comment by Andrew Morrow (Inactive) [ 28/Jul/21 ]

ali.molaei@protonmail.com - Can you please confirm regarding the AVX support in i5? If you concur, then I think we can state conclusively that the issue is that MongoDB 5.0 requires AVX support. Unfortunately, Intel has made the situation sort of confusing, in that the Pentium and Celeron branded processors do not support AVX, whether or not they are newer than Sandy Bridge, up until the ones based on Tiger Lake. We will be working to clarify our documentation to make it clearer that the real requirement is AVX (along with any other instruction extensions available on "true" Sandy Bridge; specifically, those that GCC 8 may emit when building with -march=sandybridge).

Comment by Andrew Morrow (Inactive) [ 27/Jul/21 ]

ali.molaei@protonmail.com - The link for your i5 states that it supports AVX (see the entry Instruction Set Extensions), as does the i7. So, I'm not sure why your table shows the i5 as not supporting AVX. Can you please double check that result?

Comment by Ali Molaei [ 27/Jul/21 ]

I've got some updates:

I tested/got results on more IvyBridge CPUs,
i5-3210M and i7-3540M and my old Celeron 1005M

Mongo v5.0.1 works on i5 and i7 not on the celeron, even though it's IvyBrdige

these CPU flags differences are:

Celeron | i5 | i7

--------------------------

NULL | NULL | avx

NULL | NULL | f16c

NULL | NULL | rdrand

NULL | NULL | ida

NULL | aes    | aes

NULL | smx   | smx

 

So I think aes and smx are needed in version 5?

Comment by Ali Molaei [ 24/Jul/21 ]

Sorry for commenting back late, I am using an Ivy Bridge (2012) Intel processor, (I think it's newer than Sandy Bridge (2011)), so It must be something else

Comment by Spencer Brown [ 18/Jul/21 ]

One possibility is that MongoDB 5.0 requires a minimum x86_64 microarchitecture version. Please check that the CPU meets these requirements from our Production Notes:

  • For Intel x86_64, MongoDB requires Sandy Bridge or later.
  • For AMD x86_64, MongoDB requires Bulldozer or later.
Generated at Thu Feb 08 05:45:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.