mongod start failed after unclean shutdown

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Incomplete
    • Priority: Major - P3
    • None
    • Affects Version/s: 3.0.6
    • Component/s: Admin, WiredTiger
    • None
    • ALL
    • Hide

      try repair but not working

      Show
      try repair but not working
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      mongodb start failed.

      2015-09-24T11:25:24.047+0530 I CONTROL  ***** SERVER RESTARTED *****
      2015-09-24T11:25:24.072+0530 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2015-09-24T11:25:25.528+0530 E STORAGE  [initandlisten] WiredTiger (0) [1443074125:528380][4951:0x7fdb932a8c80], file:sizeStorer.wt, session.open_cursor: read checksum error for 4096B block at offset 12288: block header checksum of 1702521171 doesn't match expected checksum of 1011620257
      2015-09-24T11:25:25.531+0530 E STORAGE  [initandlisten] WiredTiger (0) [1443074125:528469][4951:0x7fdb932a8c80], file:sizeStorer.wt, session.open_cursor: sizeStorer.wt: encountered an illegal file format or internal value
      2015-09-24T11:25:25.531+0530 E STORAGE  [initandlisten] WiredTiger (-31804) [1443074125:531883][4951:0x7fdb932a8c80], file:sizeStorer.wt, session.open_cursor: the process must exit and restart: WT_PANIC: WiredTiger library panic
      2015-09-24T11:25:25.531+0530 I -        [initandlisten] Fatal Assertion 28558
      2015-09-24T11:25:26.309+0530 I CONTROL  [initandlisten] 
       0xf5c569 0xefb431 0xedffb1 0xd8620a 0x138fe69 0x1390025 0x13904c4 0x12e081e 0x12e0d83 0x12ddb13 0x12e1aa6 0x12fb601 0x132564b 0x138f00b 0x138f49a 0x13329a1 0x138d0fa 0x13440e3 0x138cf41 0x138d355 0xd82099 0xd715a6 0xd6f258 0xa7dd5d 0x808872 0x7d6bc9 0x7fdb9186caf5 0x806689
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"B5C569"},{"b":"400000","o":"AFB431"},{"b":"400000","o":"ADFFB1"},{"b":"400000","o":"98620A"},{"b":"400000","o":"F8FE69"},{"b":"400000","o":"F90025"},{"b":"400000","o":"F904C4"},{"b":"400000","o":"EE081E"},{"b":"400000","o":"EE0D83"},{"b":"400000","o":"EDDB13"},{"b":"400000","o":"EE1AA6"},{"b":"400000","o":"EFB601"},{"b":"400000","o":"F2564B"},{"b":"400000","o":"F8F00B"},{"b":"400000","o":"F8F49A"},{"b":"400000","o":"F329A1"},{"b":"400000","o":"F8D0FA"},{"b":"400000","o":"F440E3"},{"b":"400000","o":"F8CF41"},{"b":"400000","o":"F8D355"},{"b":"400000","o":"982099"},{"b":"400000","o":"9715A6"},{"b":"400000","o":"96F258"},{"b":"400000","o":"67DD5D"},{"b":"400000","o":"408872"},{"b":"400000","o":"3D6BC9"},{"b":"7FDB9184B000","o":"21AF5"},{"b":"400000","o":"406689"}],"processInfo":{ "mongodbVersion" : "3.0.6", "gitVersion" : "1ef45a23a4c5e3480ac919b28afcba3c615488f2", "uname" : { "sysname" : "Linux", "release" : "3.10.0-229.11.1.el7.x86_64", "version" : "#1 SMP Thu Aug 6 01:06:18 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "2C8EE1A0F536BE67BDBA4481FE55CEEE8AA46950" }, { "b" : "7FFC63EFE000", "elfType" : 3, "buildId" : "3D2E7F6E0FC5432E542D442752748EF6F5A3BA17" }, { "b" : "7FDB92E8B000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "12F30315D4F4A2FE58B1977405C8B5515861E66B" }, { "b" : "7FDB92C1E000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "BB96EE99138B19FECDAB55E80A1728B648ECAD50" }, { "b" : "7FDB92837000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "B154203FB7C05AEE29D5D6F6C000305191209FE4" }, { "b" : "7FDB9262F000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "7376A07360DC57189A8F92B20AA4AA1CAEA80551" }, { "b" : "7FDB9242B000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "4DFEE4EA9AE8FDD4C71BD4CCC0727222F19DF810" }, { "b" : "7FDB92124000", "path" : "/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "405EACD649720B8668FFBBA197CBF030A7EF6296" }, { "b" : "7FDB91E22000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "A1AA62B29765BE03A36BF927B047EEEF8696EEC6" }, { "b" : "7FDB91C0C000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "5D3D7256AE68BCFF41E312A24825ED80ECA88A73" }, { "b" : "7FDB9184B000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "C31FFE7942BFD77B2FCA8F9BD5709D387A86D3BC" }, { "b" : "7FDB930A7000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9866E1D2BA61EBB4CE4F009FACDAACC24EF3B804" }, { "b" : "7FDB915FF000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "34672D541C8C9C5C1C25CB4F3F332CC9D3E604AD" }, { "b" : "7FDB9131C000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "45CB7F6CD322F5B55FF8B635F7EC1578631CCAEA" }, { "b" : "7FDB91118000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7FDB90EE6000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "23A2D854538903E2B84EF0882046DD95522C8B59" }, { "b" : "7FDB90CD0000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "E45643F27F3B3E960F3691AFC6EC27A98EF7B46B" }, { "b" : "7FDB90AC1000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "F4A3D5E7E23F871751CA8F250421F8CF83447AD2" }, { "b" : "7FDB908BD000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FDB906A3000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "AC596E865AF0D14B10F7B707F47D2031AD6D68DC" }, { "b" : "7FDB9047E000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "82FF6B18E1E42825CC2D060F969479AD4AF2F62C" }, { "b" : "7FDB9021D000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "298B19C64B19995F2AA4DA7B852E90BA5302F630" }, { "b" : "7FDB8FFF8000", "path" : "/lib64/liblzma.so.5", "elfType" : 3, "buildId" : "218D03D1F6CF1A099A4D467B5E8ECF4F2BF45750" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5c569]
       mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefb431]
       mongod(_ZN5mongo13fassertFailedEi+0x61) [0xedffb1]
       mongod(+0x98620A) [0xd8620a]
       mongod(__wt_eventv+0x489) [0x138fe69]
       mongod(__wt_err+0x95) [0x1390025]
       mongod(__wt_panic+0x24) [0x13904c4]
       mongod(__wt_block_extlist_read+0x6E) [0x12e081e]
       mongod(__wt_block_extlist_read_avail+0x33) [0x12e0d83]
       mongod(__wt_block_checkpoint_load+0x193) [0x12ddb13]
       mongod(+0xEE1AA6) [0x12e1aa6]
       mongod(__wt_btree_open+0xAB1) [0x12fb601]
       mongod(__wt_conn_btree_get+0x19B) [0x132564b]
       mongod(__wt_session_get_btree+0x41B) [0x138f00b]
       mongod(__wt_session_get_btree_ckpt+0xBA) [0x138f49a]
       mongod(__wt_curfile_open+0xE1) [0x13329a1]
       mongod(__wt_open_cursor+0x26A) [0x138d0fa]
       mongod(__wt_curtable_open+0x2E3) [0x13440e3]
       mongod(__wt_open_cursor+0xB1) [0x138cf41]
       mongod(+0xF8D355) [0x138d355]
       mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKSs+0xA9) [0xd82099]
       mongod(_ZN5mongo18WiredTigerKVEngineC1ERKSsS2_bb+0x546) [0xd715a6]
       mongod(+0x96F258) [0xd6f258]
       mongod(_ZN5mongo23GlobalEnvironmentMongoD22setGlobalStorageEngineERKSs+0x30D) [0xa7dd5d]
       mongod(_ZN5mongo13initAndListenEi+0x422) [0x808872]
       mongod(main+0x139) [0x7d6bc9]
       libc.so.6(__libc_start_main+0xF5) [0x7fdb9186caf5]
       mongod(+0x406689) [0x806689]
      -----  END BACKTRACE  -----
      2015-09-24T11:25:26.309+0530 I -        [initandlisten] 
      
      ***aborting after fassert() failure
      

      I am having worst experience with mongodb and wiredTiger, wired tiger files got corrupted very frequently on unusual shut down of system, repair process not working, and u people are not telling what is the process process of repairing wired tiger files. It is such a bad experience of using a storage engine.

      I think I should switch to MMap, that is comparetively better than this.

        1. WiredTiger
          0.0 kB
          naveen
        2. WiredTiger.basecfg
          0.5 kB
          naveen
        3. WiredTiger.turtle
          0.9 kB
          naveen
        4. WiredTiger.wt
          756 kB
          naveen
        5. _mdb_catalog.wt
          40 kB
          naveen
        6. sizeStorer.wt
          52 kB
          naveen
        7. storage.bson
          0.1 kB
          naveen
        8. newfiles.tgz
          45 kB
          Ramon Fernandez

            Assignee:
            Susan LoVerso (Inactive)
            Reporter:
            naveen
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: