Details
-
Bug
-
Resolution: Done
-
Major - P3
-
None
-
4.0.2
-
None
-
None
-
ALL
Description
Today when I start my computer, MongoDB does't not start any more. Manually runing its related systemd service (I would describe later), the error logs said:
sudo systemctl status mongo.service
|
● mongo.service - MongoDB
|
Loaded: loaded (/etc/systemd/system/mongo.service; enabled; vendor preset: enabled)
|
Active: failed (Result: core-dump) since Tue 2018-10-02 09:49:41 +07; 424ms ago |
Process: 6022 ExecStart=/storage/mongodb/bin/mongod --dbpath /storage/mongodb/data (code=dumped, signal=SEGV) |
Main PID: 6022 (code=dumped, signal=SEGV)Oct 02 09:49:41 u1804d mongod[6022]: mongod(+0xAB80BD) [0x5568f56e30bd] |
Oct 02 09:49:41 u1804d mongod[6022]: mongod(_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE+0xBF5) [0x5568f5d2d0b5] |
Oct 02 09:49:41 u1804d mongod[6022]: mongod(+0x988F56) [0x5568f55b3f56] |
Oct 02 09:49:41 u1804d mongod[6022]: mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0x873) [0x5568f5699f73] |
Oct 02 09:49:41 u1804d mongod[6022]: mongod(main+0x9) [0x5568f5630f99] |
Oct 02 09:49:41 u1804d mongod[6022]: libc.so.6(__libc_start_main+0xE7) [0x7f6ac0835b97] |
Oct 02 09:49:41 u1804d mongod[6022]: mongod(_start+0x2A) [0x5568f5698a2a] |
Oct 02 09:49:41 u1804d mongod[6022]: ----- END BACKTRACE ----- |
Oct 02 09:49:41 u1804d systemd[1]: mongo.service: Main process exited, code=dumped, status=11/SEGV |
Oct 02 09:49:41 u1804d systemd[1]: mongo.service: Failed with result 'core-dump'. |
|
If I try to run it directy, it says:
/storage/mongodb/bin/mongod --dbpath /storage/mongodb/data
|
2018-10-02T09:53:15.992+0700 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none' |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] MongoDB starting : pid=6111 port=27017 dbpath=/storage/mongodb/data 64-bit host=u1804d |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] db version v4.0.2 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] git version: fc1573ba18aee42f97a3bb13b67af7d837826b47 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.0g 2 Nov 2017 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] allocator: tcmalloc |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] modules: none |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] build environment: |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] distmod: ubuntu1804 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] distarch: x86_64 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] target_arch: x86_64 |
2018-10-02T09:53:15.999+0700 I CONTROL [initandlisten] options: { storage: { dbPath: "/storage/mongodb/data" } } |
2018-10-02T09:53:15.999+0700 I STORAGE [initandlisten] Detected data files in /storage/mongodb/data created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. |
2018-10-02T09:53:15.999+0700 I STORAGE [initandlisten] |
2018-10-02T09:53:15.999+0700 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine |
2018-10-02T09:53:15.999+0700 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem |
2018-10-02T09:53:15.999+0700 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3436M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress), |
2018-10-02T09:53:16.683+0700 I STORAGE [initandlisten] WiredTiger message [1538448796:683576][6111:0x7f3b99134a00], txn-recover: Main recovery loop: starting at 487/256 |
2018-10-02T09:53:16.683+0700 I STORAGE [initandlisten] WiredTiger message [1538448796:683829][6111:0x7f3b99134a00], txn-recover: Recovering log 487 through 488 |
2018-10-02T09:53:16.760+0700 I STORAGE [initandlisten] WiredTiger message [1538448796:760168][6111:0x7f3b99134a00], txn-recover: Recovering log 488 through 488 |
2018-10-02T09:53:16.807+0700 I STORAGE [initandlisten] WiredTiger message [1538448796:807688][6111:0x7f3b99134a00], txn-recover: Set global recovery timestamp: 0 |
2018-10-02T09:53:17.026+0700 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0) |
2018-10-02T09:53:17.063+0700 E STORAGE [initandlisten] WiredTiger error (2) [1538448797:63312][6111:0x7f3b99134a00], file:collection-30-2623056915678715117.wt, WT_SESSION.open_cursor: __posix_open_file, 711: /storage/mongodb/data/collection-30-2623056915678715117.wt: handle-open: open: No such file or directory Raw: [1538448797:63312][6111:0x7f3b99134a00], file:collection-30-2623056915678715117.wt, WT_SESSION.open_cursor: __posix_open_file, 711: /storage/mongodb/data/collection-30-2623056915678715117.wt: handle-open: open: No such file or directory |
2018-10-02T09:53:17.063+0700 E STORAGE [initandlisten] no cursor for uri: table:collection-30-2623056915678715117 |
2018-10-02T09:53:17.063+0700 F - [initandlisten] Invalid access at address: 0x58 |
2018-10-02T09:53:17.072+0700 F - [initandlisten] Got signal: 11 (Segmentation fault). |
0x55cb0a04fae1 0x55cb0a04ecf9 0x55cb0a04f366 0x7f3b977ef890 0x55cb08772418 0x55cb0877daad 0x55cb0876488c 0x55cb088e8108 0x55cb088e2885 0x55cb088e354d 0x55cb0874f0bd 0x55cb08d990b5 0x55cb0861ff56 0x55cb08705f73 0x55cb0869cf99 0x7f3b9740db97 0x55cb08704a2a |
----- BEGIN BACKTRACE -----
|
{"backtrace":[{"b":"55CB07C97000","o":"23B8AE1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55CB07C97000","o":"23B7CF9"},{"b":"55CB07C97000","o":"23B8366"},{"b":"7F3B977DD000","o":"12890"},{"b":"55CB07C97000","o":"ADB418","s":"_ZN5mongo31WiredTigerRecordStoreCursorBase4nextEv"},{"b":"55CB07C97000","o":"AE6AAD","s":"_ZN5mongo21WiredTigerRecordStore19postConstructorInitEPNS_16OperationContextE"},{"b":"55CB07C97000","o":"ACD88C","s":"_ZN5mongo18WiredTigerKVEngine21getGroupedRecordStoreEPNS_16OperationContextENS_10StringDataES3_RKNS_17CollectionOptionsENS_8KVPrefixE"},{"b":"55CB07C97000","o":"C51108","s":"_ZN5mongo26KVDatabaseCatalogEntryBase14initCollectionEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb"},{"b":"55CB07C97000","o":"C4B885","s":"_ZN5mongo15KVStorageEngine11loadCatalogEPNS_16OperationContextE"},{"b":"55CB07C97000","o":"C4C54D","s":"_ZN5mongo15KVStorageEngineC2EPNS_8KVEngineERKNS_22KVStorageEngineOptionsESt8functionIFSt10unique_ptrINS_26KVDatabaseCatalogEntryBaseESt14default_deleteIS8_EENS_10StringDataEPS0_EE"},{"b":"55CB07C97000","o":"AB80BD"},{"b":"55CB07C97000","o":"11020B5","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE"},{"b":"55CB07C97000","o":"988F56"},{"b":"55CB07C97000","o":"A6EF73","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},{"b":"55CB07C97000","o":"A05F99","s":"main"},{"b":"7F3B973EC000","o":"21B97","s":"__libc_start_main"},{"b":"55CB07C97000","o":"A6DA2A","s":"_start"}],"processInfo":{ "mongodbVersion" : "4.0.2", "gitVersion" : "fc1573ba18aee42f97a3bb13b67af7d837826b47", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.15.0-33-generic", "version" : "#36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "55CB07C97000", "elfType" : 3, "buildId" : "8F771DD94C444FE18EFA84CA664CE8A0E6100804" }, { "b" : "7FFEC19E6000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "FFC34FD287857F2EB9923E5FF16B73F3BF3F7B39" }, { "b" : "7F3B98CBB000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "33F6C90EE440B440A9ADBD0124BDCAE2FFE7C47B" }, { "b" : "7F3B98AA0000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "390E9CC4C215314B6D8ADE6D6E28F8518418039C" }, { "b" : "7F3B98628000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1", "elfType" : 3, "buildId" : "DA6345B41E06A9001B5C51DEAF0784BE9D49810D" }, { "b" : "7F3B983BE000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", "elfType" : 3, "buildId" : "9787FF1FFFE8D82268D89F6BFA3AF07B3E9D140F" }, { "b" : "7F3B981BA000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "25AD56E902E23B490A9CCDB08A9744D89CB95BCC" }, { "b" : "7F3B97FB2000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "9826FBDF57ED7D6965131074CB3C08B1009C1CD8" }, { "b" : "7F3B97C14000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "A33761AB8FB485311B3C85BF4253099D7CABE653" }, { "b" : "7F3B979FC000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "92E0BE1929D28508CF9C6D5754C7EB48C12255B3" }, { "b" : "7F3B977DD000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "28C6AADE70B2D40D1F0F3D0A1A0CAD1AB816448F" }, { "b" : "7F3B973EC000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B417C0BA7CC5CF06D1D1BED6652CEDB9253C60D0" }, { "b" : "7F3B98F3A000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "64DF1B961228382FE18684249ED800AB1DCEAAD4" }, { "b" : "7F3B971C7000", "path" : "/usr/lib/x86_64-linux-gnu/libnghttp2.so.14", "elfType" : 3, "buildId" : "4F00E5207693FDC249DA42EC6472ACA6A7B929AE" }, { "b" : "7F3B96FAA000", "path" : "/usr/lib/x86_64-linux-gnu/libidn2.so.0", "elfType" : 3, "buildId" : "BA5BF9A5C44F48C647E9D8270A5421AE81CCAD61" }, { "b" : "7F3B96D8E000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "69465D8AA6B19086ABF2455A703F9168BF82A69F" }, { "b" : "7F3B96B80000", "path" : "/usr/lib/x86_64-linux-gnu/libpsl.so.5", "elfType" : 3, "buildId" : "CDAF1F1946846941F9D06414EC8C812D131A168E" }, { "b" : "7F3B96935000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "451E2EC02C8F974B21D1CE47F5D2C21A21C16A9F" }, { "b" : "7F3B966E3000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "0CDDF9D1F87FE7FF07D73DBE1D7CB57EE5BA802C" }, { "b" : "7F3B964D5000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "DD2893FC0AA3C58798F805D4C88EE086564D2EEB" }, { "b" : "7F3B962B8000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "EF3E006DFE3132A41D4D4DC0E407D6EA658E11C4" }, { "b" : "7F3B95F3A000", "path" : "/usr/lib/x86_64-linux-gnu/libunistring.so.2", "elfType" : 3, "buildId" : "62A44D226F3A74CF720305A52D3E876E214C248A" }, { "b" : "7F3B95BD5000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "F67AAC89381F6ACFE9979B24D4002668C4F70ABE" }, { "b" : "7F3B959A1000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "842BDF0B0EAAB82E19F1EABFC38769F4040FBE31" }, { "b" : "7F3B9576B000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "C20D4B3BA13FCDCC3BF6857689BA9FC70BE3F6A5" }, { "b" : "7F3B954EA000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "D40EA9B5EC5BC46799E4A412319617BD38BE9341" }, { "b" : "7F3B95214000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "AFF9A2A2B6ACB066BF450CC4A0772B9DC04299CB" }, { "b" : "7F3B94FE2000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "04AADCB371B97A570756D69A0D5B3B0E6737E349" }, { "b" : "7F3B94DDE000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "9E885FB7F54DA464A3D2A978C71C1178A6EC7512" }, { "b" : "7F3B94BD3000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "27A5983855C50A37EFA526BB12EA48CDAE3B5FCA" }, { "b" : "7F3B949B8000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "ABB7E3F40302E6509DAD1F91DFB1F04B6A5FD072" }, { "b" : "7F3B94777000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "A1A98DB481968073636BBAECB561A3EA8ED198AE" }, { "b" : "7F3B94448000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "8DBD451EA5651283905E16FA7DFA9908688893A3" }, { "b" : "7F3B94235000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "6036B89A3BB671B32E01464C0C82BFA016186352" }, { "b" : "7F3B94031000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "F463E107B099910463BC32E837C73D341A52C27B" }, { "b" : "7F3B93E28000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "C2376C5B831991591F1A67B976758185F86896D8" }, { "b" : "7F3B93B9B000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "69BDEE5FA0FEEDF317308BE850F78761861D520A" }, { "b" : "7F3B938F9000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "315D74995AAA32DE4D15BA25F335066988B1B230" }, { "b" : "7F3B936C3000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "6673972A1C24A89EBAFBAE696188A4CB26C6DDEB" }, { "b" : "7F3B934AD000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "430827C33259C12248CF44B91A9A9821114376F5" }, { "b" : "7F3B932A5000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "3555B5F599C9787DFDDBF9E8DF6F706B9044D985" }, { "b" : "7F3B9307C000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType" : 3, "buildId" : "93A0931B1C2818F0EA224CE6FE5E31E84A9B55BB" }, { "b" : "7F3B92E6D000", "path" : "/usr/lib/x86_64-linux-gnu/libheimbase.so.1", "elfType" : 3, "buildId" : "669D4CCE42FA4382796EFFCF0C16F459F4382C4C" }, { "b" : "7F3B92C23000", "path" : "/usr/lib/x86_64-linux-gnu/libhx509.so.5", "elfType" : 3, "buildId" : "4B80C543356EE0AF9039EFE7C9EA1CC1F74C426A" }, { "b" : "7F3B9291A000", "path" : "/usr/lib/x86_64-linux-gnu/libsqlite3.so.0", "elfType" : 3, "buildId" : "9FD7B69263297314C83BB205234E182BD4D8B86A" }, { "b" : "7F3B926E2000", "path" : "/lib/x86_64-linux-gnu/libcrypt.so.1", "elfType" : 3, "buildId" : "810686AF0D5FD350A4FB1CC4B5AFF44A05C102CB" } ] }} |
mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55cb0a04fae1] |
mongod(+0x23B7CF9) [0x55cb0a04ecf9] |
mongod(+0x23B8366) [0x55cb0a04f366] |
libpthread.so.0(+0x12890) [0x7f3b977ef890] |
mongod(_ZN5mongo31WiredTigerRecordStoreCursorBase4nextEv+0x228) [0x55cb08772418] |
mongod(_ZN5mongo21WiredTigerRecordStore19postConstructorInitEPNS_16OperationContextE+0xCD) [0x55cb0877daad] |
mongod(_ZN5mongo18WiredTigerKVEngine21getGroupedRecordStoreEPNS_16OperationContextENS_10StringDataES3_RKNS_17CollectionOptionsENS_8KVPrefixE+0x27C) [0x55cb0876488c] |
mongod(_ZN5mongo26KVDatabaseCatalogEntryBase14initCollectionEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb+0x568) [0x55cb088e8108] |
mongod(_ZN5mongo15KVStorageEngine11loadCatalogEPNS_16OperationContextE+0xA95) [0x55cb088e2885] |
mongod(_ZN5mongo15KVStorageEngineC2EPNS_8KVEngineERKNS_22KVStorageEngineOptionsESt8functionIFSt10unique_ptrINS_26KVDatabaseCatalogEntryBaseESt14default_deleteIS8_EENS_10StringDataEPS0_EE+0x22D) [0x55cb088e354d] |
mongod(+0xAB80BD) [0x55cb0874f0bd] |
mongod(_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE+0xBF5) [0x55cb08d990b5] |
mongod(+0x988F56) [0x55cb0861ff56] |
mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0x873) [0x55cb08705f73] |
mongod(main+0x9) [0x55cb0869cf99] |
libc.so.6(__libc_start_main+0xE7) [0x7f3b9740db97] |
mongod(_start+0x2A) [0x55cb08704a2a] |
----- END BACKTRACE -----
|
Segmentation fault (core dumped)
|
Sorry for the very long stack trace inserted, I hope they make some sense.
For setting up MongoDB, I often do the same steps as below:
- Create a location for database engine/storage
- Download .tgz file, extract into its location
- Create systemd service file to run it as service
I work on Ubuntu 18.04, minimal version with XFCE DE, gcc/g++ v7.3.0
Here is my installation script for MongoDB 4.0.2 that is occurring issue:
export MONGO_VERSION=4.0.2 |
export MONGO_DOWNLOAD_URL=https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-$MONGO_VERSION.tgz |
export MONGO_DIR=/storage/mongodb
|
sudo apt install libcurl3 openssl
|
sudo mkdir -p $MONGO_DIR/data
|
sudo chown ${USER} -R $MONGO_DIR
|
wget "$MONGO_DOWNLOAD_URL" -O mongo.tar.tgz |
tar -zxvf mongo.tar.tgz -C $MONGO_DIR --strip-components=1 |
And mongo.service in /etc/systemd/system
[Unit]
|
Description=MongoDB
|
After=network.target[Service]
|
User=ndaidong
|
WorkingDirectory=/storage/mongodb
|
ExecStart=/storage/mongodb/bin/mongod --dbpath /storage/mongodb/data
|
Restart=always
|
RestartSec=500ms
|
StartLimitInterval=0[Install] |
WantedBy=multi-user.target
|
It works like as charm until today. Nothing special changes in my system.
What's wrong with my configuration? And how to get it works again?
Thank you for any advice.