[SERVER-15610] Failure to start daemon process mongod that built with rocksdb experimental. Created: 13/Oct/14  Updated: 28/Apr/15  Resolved: 04/Dec/14

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.7.8, 2.8.0-rc1
Fix Version/s: None

Type: Question Priority: Critical - P2
Reporter: rchatsiri Assignee: Matt Kangas
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File mongod_rocksdb_error.txt    
Backwards Compatibility: Fully Compatible
Participants:

 Description   

System using Linux OS. Ubuntu distributed, Kernel version 3.13.0-24-generic-64 bits. G++ version 4.7.3. After deamon process name mongod run with command line.

./mongod --storageEngine=rocksExperiment --dbpath ~/rocksdbmongo/

Programs throw exceptions as blow

(gdb) run --storageEngine=rocksExperiment --dbpath ~/rocksdbmongo/
Starting program: /mongo/build/linux2/c++11_on/mongo/mongod --storageEngine=rocksExperiment --dbpath ~/rocksdbmongo/
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff64d7700 (LWP 16183)]
2014-10-12T22:38:52.781-0400 I -        [initandlisten] MongoDB starting : pid=16182 port=27017 dbpath=/home/rch/rocksdbmongo/ 64-bit host=vpnrch
2014-10-12T22:38:52.781-0400 I -        [initandlisten]
2014-10-12T22:38:52.782-0400 I -        [initandlisten] ** NOTE: This is a development version (2.7.8-pre-) of MongoDB.
2014-10-12T22:38:52.782-0400 I -        [initandlisten] **       Not recommended for production.
2014-10-12T22:38:52.782-0400 I -        [initandlisten]
2014-10-12T22:38:52.782-0400 I -        [initandlisten] db version v2.7.8-pre-
2014-10-12T22:38:52.782-0400 I -        [initandlisten] git version: bc8c4b81164c17f95d01262e6aeb22f934e5ed8f
2014-10-12T22:38:52.782-0400 I -        [initandlisten] build info: Linux vpnrch 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2014-10-12T22:38:52.782-0400 I -        [initandlisten] allocator: tcmalloc
2014-10-12T22:38:52.782-0400 I -        [initandlisten] options: { storage: { dbPath: "/home/rch/rocksdbmongo/", engine: "rocksExperiment" } }
2014-10-12T22:38:52.783-0400 I NETWORK  [initandlisten] couldn't unlink socket file /tmp/mongodb-27017.sockerrno:1 Operation not permitted skipping
[New Thread 0x7ffff5cd6700 (LWP 16184)]
[New Thread 0x7ffff54d5700 (LWP 16185)]
[New Thread 0x7ffff4cd4700 (LWP 16186)]
[New Thread 0x7ffff44d3700 (LWP 16187)]
[New Thread 0x7ffff3cd2700 (LWP 16188)]
[New Thread 0x7ffff34d1700 (LWP 16189)]
[New Thread 0x7ffff2cd0700 (LWP 16190)]
[New Thread 0x7ffff24cf700 (LWP 16191)]
[New Thread 0x7ffff1cce700 (LWP 16192)]
[New Thread 0x7ffff14cd700 (LWP 16193)]
[New Thread 0x7ffff0ccc700 (LWP 16194)]
[New Thread 0x7ffff04cb700 (LWP 16195)]
[New Thread 0x7fffefcca700 (LWP 16196)]
[New Thread 0x7fffef4c9700 (LWP 16197)]
[New Thread 0x7fffeecc8700 (LWP 16198)]
[New Thread 0x7fffee4c7700 (LWP 16199)]
2014-10-12T22:38:52.808-0400 E STORAGE  [initandlisten] rocks error: NotFound:
2014-10-12T22:38:52.808-0400 I -        [initandlisten] Invariant failure false src/mongo/db/storage/rocks/rocks_engine.cpp 601
2014-10-12T22:38:52.818-0400 I -        [initandlisten]
 0xf6ea53 0xf21e4b 0xf08479 0xdb3dc5 0xdb5d47 0xdbb49a 0xaf30d8 0x8a8f1c 0x88339b 0x7ffff67c2ec5 0x8a764a
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B6EA53"},{"b":"400000","o":"B21E4B"},{"b":"400000","o":"B08479"},{"b":"400000","o":"9B3DC5"},{"b":"400000","o":"9B5D47"},{"b":"400000","o":"9BB49A"},{"b":"400000","o":"6F30D8"},{"b":"400000","o":"4A8F1C"},{"b":"400000","o":"48339B"},{"b":"7FFFF67A1000","o":"21EC5"},{"b":"400000","o":"4A764A"}],"processInfo":{ "mongodbVersion" : "2.7.8-pre-", "gitVersion" : "bc8c4b81164c17f95d01262e6aeb22f934e5ed8f", "uname" : { "sysname" : "Linux", "release" : "3.13.0-24-generic", "version" : "#46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "B6E4AB0DE8FF639611D6169A3A670C2EFB6026E9" }, { "b" : "7FFFF88FA000", "elfType" : 3, "buildId" : "6755FAD2CADACDF1667E5B57FF1EDFC28DD1C976" }, { "b" : "7FFFF7BC1000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "7EE9BC42787A0F9A793BDBFCB0671FD12C52BC2A" }, { "b" : "7FFFF79B1000", "path" : "/lib/x86_64-linux-gnu/libbz2.so.1.0", "elfType" : 3, "buildId" : "E1031DDBFFE20367E874B7093EEC0C8D9F3B43F6" }, { "b" : "7FFFF7793000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "FE662C4D7B14EE804E0C1902FB55218A106BC5CB" }, { "b" : "7FFFF758B000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "92FCF41EFE012D6186E31A59AD05BDBB487769AB" }, { "b" : "7FFFF7387000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "C1AE4CB7195D337A77A3C689051DABAA3980CA0C" }, { "b" : "7FFFF7083000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "19EFDDAB11B3BF5C71570078C59F91CF6592CE9E" }, { "b" : "7FFFF6D7D000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "574C6350381DA194C00FF555E0C1784618C05569" }, { "b" : "7FFFF6B67000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "CC0D578C2E0D86237CA7B0CE8913261C506A629A" }, { "b" : "7FFFF67A1000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B515361E474796AF29DE9992B76A97CFFB39B2A7" }, { "b" : "7FFFF7DDA000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9F00581AB3C73E3AEA35995A0C50D24D59A01D47" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xf6ea53]
 mongod(_ZN5mongo10logContextEPKc+0xDB) [0xf21e4b]
 mongod(_ZN5mongo15invariantFailedEPKcS1_j+0x99) [0xf08479]
 mongod(_ZN5mongo11RocksEngine14_createEntriesERKSt6vectorIN7rocksdb22ColumnFamilyDescriptorESaIS3_EES1_IPNS2_18ColumnFamilyHandleESaIS9_EE+0x8C5) [0xdb3dc5]
 mongod(_ZN5mongo11RocksEngineC1ERKSs+0x327) [0xdb5d47]
 mongod(+0x9BB49A) [0xdbb49a]
 mongod(_ZN5mongo23GlobalEnvironmentMongoD22setGlobalStorageEngineERKSs+0xF8) [0xaf30d8]
 mongod(_ZN5mongo13initAndListenEi+0x50C) [0x8a8f1c]
 mongod(main+0x11B) [0x88339b]
 libc.so.6(__libc_start_main+0xF5) [0x7ffff67c2ec5]
 mongod(+0x4A764A) [0x8a764a]
-----  END BACKTRACE  -----
 
Program received signal SIGTRAP, Trace/breakpoint trap.
0x00007ffff77a320b in raise (sig=5) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
37      ../nptl/sysdeps/unix/sysv/linux/pt-raise.c: No such file or directory.
 
Backtrack error in line of code with gdb. 
(gdb) bt
#0  0x00007ffff77a320b in raise (sig=5) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
#1  0x0000000000f1a2d7 in mongo::mongo_breakpoint () at src/mongo/util/debug_util.cpp:58
#2  0x0000000000f0847e in breakpoint () at src/mongo/util/debug_util.h:73
#3  mongo::invariantFailed (msg=msg@entry=0x14fb795 "false", file=file@entry=0x147f610 "src/mongo/db/storage/rocks/rocks_engine.cpp", line=line@entry=601)
    at src/mongo/util/assert_util.cpp:143
#4  0x0000000000db3dc5 in mongo::RocksEngine::_createEntries (this=this@entry=0x2a021a0, families=..., handles=...) at src/mongo/db/storage/rocks/rocks_engine.cpp:601
#5  0x0000000000db5d47 in mongo::RocksEngine::RocksEngine (this=0x2a021a0, path=...) at src/mongo/db/storage/rocks/rocks_engine.cpp:109
#6  0x0000000000dbb49a in mongo::(anonymous namespace)::RocksFactory::create (this=<optimized out>, params=...) at src/mongo/db/storage/rocks/rocks_init.cpp:45
#7  0x0000000000af30d8 in mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine (this=0x22ba320, name=...) at src/mongo/db/global_environment_d.cpp:74
#8  0x00000000008a8f1c in _initAndListen (listenPort=27017) at src/mongo/db/db.cpp:473
#9  mongo::initAndListen (listenPort=27017) at src/mongo/db/db.cpp:580
#10 0x000000000088339b in mongoDbMain (envp=<optimized out>, argv=0x7fffffffeb68, argc=4) at src/mongo/db/db.cpp:816
#11 main (argc=4, argv=0x7fffffffeb68, envp=<optimized out>) at src/mongo/db/db.cpp:629

I didn't know point to investigate this error.

R.Chatsiri



 Comments   
Comment by Matt Kangas [ 04/Dec/14 ]

rchatsiri, I see that you built against commit bc8c4b8. This is from October 8 and it 965 commits behind r2.8.0-rc2. RocksDB support is very much a work in progress (see SERVER-15744). If you would like to experiment with it, please use the latest master branches of both MongoDB and RocksDB.

PS: Ramon, --storageEngine=rocksExperiment is only valid on a mongod which has been compiled with RocksDB support. We don't include it in our standard builds. That's why it reports "unknown storage engine".

Comment by Ramon Fernandez Marina [ 04/Dec/14 ]

I get a different error on current master, but I still can't start mongod with rocksdb:

$ mongod --storageEngine=rocksExperiment --dbpath data
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] MongoDB starting : pid=27192 port=27017 dbpath=data 64-bit host=tab
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] db version v2.8.0-rc2-pre-
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] git version: 3b1d97d430a229bf57c46bc7bc4977801b5bb553
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] build info: Linux tab 3.13.0-36-generic #63-Ubuntu SMP Wed Sep 3 21:30:07 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] allocator: tcmalloc
2014-12-03T19:11:57.777-0500 I CONTROL  [initandlisten] options: { storage: { dbPath: "data", engine: "rocksExperiment" } }
2014-12-03T19:11:57.777-0500 I STORAGE  [initandlisten] exception in initAndListen: 18656 Cannot start server with an unknown storage engine: rocksExperiment, terminating
2014-12-03T19:11:57.777-0500 I COMMAND  [initandlisten] dbexit:  rc: 100

Comment by rchatsiri [ 13/Oct/14 ]

I see commit log : #6e98f5003ff06369aec221c9c9dda264d4dee013 at github. My local repository contain log number SERVER-15085. Mongod error in version 2.7.8 that same issues in version 2.7.6 from reference link.

Comment by rchatsiri [ 13/Oct/14 ]

Attract file is error flow when debug with gdb.

Generated at Thu Feb 08 03:38:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.