[SERVER-23811] Cannot start MongoDB, possible WiredTiger error Created: 19/Apr/16  Updated: 27/Apr/16  Resolved: 27/Apr/16

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Swati Assignee: Susan LoVerso
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: HTML File WiredTiger     File WiredTiger.lock     File WiredTiger.turtle     File WiredTiger.wt     File WiredTigerLAS.wt     File sizeStorer.wt    
Operating System: ALL
Participants:

 Description   

I cannot start the mongo sever at all
I tried – repair and removing locks but I get this tracke

2016-04-19T12:34:58.919-0700 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] MongoDB starting : pid=4039 port=27017 dbpath=/media/Store/mtest0 64-bit host=swati-HP-Pavilion-dm4-Notebook-PC
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] db version v3.2.1
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] git version: a14d55980c2cdc565d4704a7e3ad37e4e535c1b2
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] allocator: tcmalloc
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] modules: none
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] build environment:
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten]     distmod: ubuntu1404
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten]     distarch: x86_64
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten]     target_arch: x86_64
2016-04-19T12:34:58.930-0700 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/media/Store/mtest0", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-04-19T12:34:58.971-0700 I -        [initandlisten] Detected data files in /media/Store/mtest0 created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-04-19T12:34:58.971-0700 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-04-19T12:35:01.149-0700 E STORAGE  [initandlisten] WiredTiger (-31802) [1461094501:149246][4039:0x7f0a2daccd00], txn-recover: log file journal/WiredTigerLog.0000000184 corrupted: Bad magic number 0: WT_ERROR: non-specific WiredTiger error
2016-04-19T12:35:01.149-0700 E STORAGE  [initandlisten] WiredTiger (-31804) [1461094501:149384][4039:0x7f0a2daccd00], txn-recover: the process must exit and restart: WT_PANIC: WiredTiger library panic
2016-04-19T12:35:01.149-0700 I -        [initandlisten] Fatal Assertion 28558
2016-04-19T12:35:01.149-0700 I -        [initandlisten] 
 
***aborting after fassert() failure
 
 
2016-04-19T12:35:01.175-0700 F -        [initandlisten] Got signal: 6 (Aborted).
 
 0x12ea922 0x12e9a89 0x12ea292 0x7f0a2c45b340 0x7f0a2c0bccc9 0x7f0a2c0c00d8 0x1275382 0x1074313 0x1a298cc 0x1a29a6d 0x1a29e54 0x19dd735 0x19df7be 0x1a383cf 0x19b8ee7 0x19b2239 0x105a917 0x1057b60 0xf84ee8 0x9913c1 0x94cc49 0x7f0a2c0a7ec5 0x98ef5c
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"EEA922"},{"b":"400000","o":"EE9A89"},{"b":"400000","o":"EEA292"},{"b":"7F0A2C44B000","o":"10340"},{"b":"7F0A2C086000","o":"36CC9"},{"b":"7F0A2C086000","o":"3A0D8"},{"b":"400000","o":"E75382"},{"b":"400000","o":"C74313"},{"b":"400000","o":"16298CC"},{"b":"400000","o":"1629A6D"},{"b":"400000","o":"1629E54"},{"b":"400000","o":"15DD735"},{"b":"400000","o":"15DF7BE"},{"b":"400000","o":"16383CF"},{"b":"400000","o":"15B8EE7"},{"b":"400000","o":"15B2239"},{"b":"400000","o":"C5A917"},{"b":"400000","o":"C57B60"},{"b":"400000","o":"B84EE8"},{"b":"400000","o":"5913C1"},{"b":"400000","o":"54CC49"},{"b":"7F0A2C086000","o":"21EC5"},{"b":"400000","o":"58EF5C"}],"processInfo":{ "mongodbVersion" : "3.2.1", "gitVersion" : "a14d55980c2cdc565d4704a7e3ad37e4e535c1b2", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.19.0-25-generic", "version" : "#26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "D76764D44DE9B088362776AF243199FDCF5756E8" }, { "b" : "7FFEC99C9000", "elfType" : 3, "buildId" : "01005E92EA3C0C1527396B65C7412468298D0813" }, { "b" : "7F0A2D670000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "A20EFFEC993A8441FA17F2079F923CBD04079E19" }, { "b" : "7F0A2D295000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "F000D29917E9B6E94A35A8F02E5C62846E5916BC" }, { "b" : "7F0A2D08D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "92FCF41EFE012D6186E31A59AD05BDBB487769AB" }, { "b" : "7F0A2CE89000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "C1AE4CB7195D337A77A3C689051DABAA3980CA0C" }, { "b" : "7F0A2CB85000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "D0E735DBECD63462DA114BD3F76E6EC7BB1FACCC" }, { "b" : "7F0A2C87F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "1D76B71E905CB867B27CEF230FCB20F01A3178F5" }, { "b" : "7F0A2C669000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "8D0AA71411580EE6C08809695C3984769F25725B" }, { "b" : "7F0A2C44B000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "9318E8AF0BFBE444731BB0461202EF57F7C39542" }, { "b" : "7F0A2C086000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "30C94DC66A1FE95180C3D68D2B89E576D5AE213C" }, { "b" : "7F0A2D8CF000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9F00581AB3C73E3AEA35995A0C50D24D59A01D47" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x12ea922]
 mongod(+0xEE9A89) [0x12e9a89]
 mongod(+0xEEA292) [0x12ea292]
 libpthread.so.0(+0x10340) [0x7f0a2c45b340]
 libc.so.6(gsignal+0x39) [0x7f0a2c0bccc9]
 libc.so.6(abort+0x148) [0x7f0a2c0c00d8]
 mongod(_ZN5mongo13fassertFailedEi+0x82) [0x1275382]
 mongod(+0xC74313) [0x1074313]
 mongod(__wt_eventv+0x40C) [0x1a298cc]
 mongod(__wt_err+0x8D) [0x1a29a6d]
 mongod(__wt_panic+0x24) [0x1a29e54]
 mongod(+0x15DD735) [0x19dd735]
 mongod(__wt_log_scan+0x10E) [0x19df7be]
 mongod(__wt_txn_recover+0x2CF) [0x1a383cf]
 mongod(__wt_connection_workers+0x37) [0x19b8ee7]
 mongod(wiredtiger_open+0x15F9) [0x19b2239]
 mongod(_ZN5mongo18WiredTigerKVEngineC2ERKSsS2_S2_mbbb+0x557) [0x105a917]
 mongod(+0xC57B60) [0x1057b60]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x588) [0xf84ee8]
 mongod(_ZN5mongo13initAndListenEi+0x321) [0x9913c1]
 mongod(main+0x149) [0x94cc49]
 libc.so.6(__libc_start_main+0xF5) [0x7f0a2c0a7ec5]
 mongod(+0x58EF5C) [0x98ef5c]
-----  END BACKTRACE  -----



 Comments   
Comment by Susan LoVerso [ 27/Apr/16 ]

User never replied

Comment by Susan LoVerso [ 20/Apr/16 ]

Can you also upload your mongod log file? When did mongod go down prior to the error restarting and under what circumstances?

Comment by Susan LoVerso [ 20/Apr/16 ]

swatisaoji1 can you post an ls -l of your journal directory? Also I may need to look at the journal files, so can you save a copy of them. Could you upload journal/WiredTigerLog.0000000184?

Comment by Swati [ 19/Apr/16 ]

attached the files if they help in understanding the issue , Thanks for guidance

Generated at Thu Feb 08 04:04:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.