Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-35005

file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Done
    • 3.3.0
    • None
    • None
    • Linux

    Description

      Hi,
      as many other on jira.mongodb.org I faced with the same situation. Mongodb/server has unexpected shutdown. After startup mongo print out error listed below. I've tried --repair and other suggestions that google finds, but didn't helped.

      If some could help me I would be extremely grateful.
      Thank you.

       

      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=04d1cfeeb14e
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] db version v3.3.15
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] git version: 520f5571d039b57cf9c319b49654909828971073
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1t  3 May 2016
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] allocator: tcmalloc
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] modules: none
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] build environment:
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten]     distmod: debian81
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten]     distarch: x86_64
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten]     target_arch: x86_64
      2018-05-15T18:22:34.131+0000 I CONTROL  [initandlisten] options: {}
      2018-05-15T18:22:34.135+0000 I -        [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
      2018-05-15T18:22:34.136+0000 I STORAGE  [initandlisten]
      2018-05-15T18:22:34.136+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
      2018-05-15T18:22:34.136+0000 I STORAGE  [initandlisten] See http://dochub.mongodb.org/core/prodnotes-filesystem
      2018-05-15T18:22:34.136+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=706M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2018-05-15T18:22:34.148+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1526408554:148875][1:0x7f3ab1b38cc0], file:WiredTiger.wt, connection: WiredTiger.turtle: encountered an illegal file format or internal value
      2018-05-15T18:22:34.148+0000 E STORAGE  [initandlisten] WiredTiger error (-31804) [1526408554:148970][1:0x7f3ab1b38cc0], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic
      2018-05-15T18:22:34.149+0000 I -        [initandlisten] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361
      2018-05-15T18:22:34.149+0000 I -        [initandlisten]***aborting after fassert() failure
      2018-05-15T18:22:34.190+0000 F -        [initandlisten] Got signal: 6 (Aborted). 0x55970bee2fe1 0x55970bee20d9 0x55970bee25bd 0x7f3ab07948d0 0x7f3ab040f067 0x7f3ab0410448 0x55970b1a48e5 0x55970bbce796 0x55970b1aee91 0x55970b1aef80 0x55970b1af1d8 0x55970c87ceab 0x55970c87b79e 0x55970c831440 0x55970c8b8c93 0x55970c8b9279 0x55970c8b950c 0x55970c840cd8 0x55970c8afc00 0x55970c87adfe 0x55970c87aeeb 0x55970c82dab8 0x55970bbb2b19 0x55970bbab94c 0x55970ba9c1b0 0x55970b19066c 0x55970b1b07ff 0x7f3ab03fbb45 0x55970b20c8ef
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"55970A9A0000","o":"1542FE1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55970A9A0000","o":"15420D9"},{"b":"55970A9A0000","o":"15425BD"},{"b":"7F3AB0785000","o":"F8D0"},{"b":"7F3AB03DA000","o":"35067","s":"gsignal"},{"b":"7F3AB03DA000","o":"36448","s":"abort"},{"b":"55970A9A0000","o":"8048E5","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"55970A9A0000","o":"122E796"},{"b":"55970A9A0000","o":"80EE91","s":"__wt_eventv"},{"b":"55970A9A0000","o":"80EF80","s":"__wt_err"},{"b":"55970A9A0000","o":"80F1D8","s":"__wt_panic"},{"b":"55970A9A0000","o":"1EDCEAB","s":"__wt_turtle_read"},{"b":"55970A9A0000","o":"1EDB79E","s":"__wt_metadata_search"},{"b":"55970A9A0000","o":"1E91440","s":"__wt_conn_btree_open"},{"b":"55970A9A0000","o":"1F18C93","s":"__wt_session_get_btree"},{"b":"55970A9A0000","o":"1F19279","s":"__wt_session_get_btree"},{"b":"55970A9A0000","o":"1F1950C","s":"__wt_session_get_btree_ckpt"},{"b":"55970A9A0000","o":"1EA0CD8","s":"__wt_curfile_open"},{"b":"55970A9A0000","o":"1F0FC00"},{"b":"55970A9A0000","o":"1EDADFE","s":"__wt_metadata_cursor_open"},{"b":"55970A9A0000","o":"1EDAEEB","s":"__wt_metadata_cursor"},{"b":"55970A9A0000","o":"1E8DAB8","s":"wiredtiger_open"},{"b":"55970A9A0000","o":"1212B19","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"55970A9A0000","o":"120B94C"},{"b":"55970A9A0000","o":"10FC1B0","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"55970A9A0000","o":"7F066C"},{"b":"55970A9A0000","o":"8107FF","s":"main"},{"b":"7F3AB03DA000","o":"21B45","s":"__libc_start_main"},{"b":"55970A9A0000","o":"86C8EF"}],"processInfo":{ "mongodbVersion" : "3.3.15", "gitVersion" : "520f5571d039b57cf9c319b49654909828971073", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.15.0-20-generic", "version" : "#21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "55970A9A0000", "elfType" : 3, "buildId" : "3577DE2A89C1B5B1EB059231161B733A14CCB52A" }, { "b" : "7FFDAC3BD000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "A0297FE29F8038EF50A10A26C9FCF5249CCC1184" }, { "b" : "7F3AB16C1000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "0711D67B299ACA3EA6B0B57C0DA4EC83B71787EE" }, { "b" : "7F3AB12C5000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "2E613A282B22F2B0978ACCEC1B71411F5E20CC89" }, { "b" : "7F3AB10BD000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "18650C365FBA12BA12DB680B988B7754C0F99364" }, { "b" : "7F3AB0EB9000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "1D841026FF2C5AEA25C21921FAC9C978AB465825" }, { "b" : "7F3AB0BB8000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "BBA5CF06072F15BEA36FC8B0A2C6A32BBF08C235" }, { "b" : "7F3AB09A2000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "D5FB04F64B3DAEA6D6B68B5E8B9D4D2BC1A6E1FC" }, { "b" : "7F3AB0785000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "B9DD2674891C5395C5724321158DC2EB8C7DA64B" }, { "b" : "7F3AB03DA000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "AD0CE0A6B6E201C9A8195B2D55043AFCD4F5DB00" }, { "b" : "7F3AB1922000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "44DA6E33EBAB324C1D0128F9465425AD355CC978" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55970bee2fe1]
       mongod(+0x15420D9) [0x55970bee20d9]
       mongod(+0x15425BD) [0x55970bee25bd]
       libpthread.so.0(+0xF8D0) [0x7f3ab07948d0]
       libc.so.6(gsignal+0x37) [0x7f3ab040f067]
       libc.so.6(abort+0x148) [0x7f3ab0410448]
       mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55970b1a48e5]
       mongod(+0x122E796) [0x55970bbce796]
       mongod(__wt_eventv+0x496) [0x55970b1aee91]
       mongod(__wt_err+0x9D) [0x55970b1aef80]
       mongod(__wt_panic+0x24) [0x55970b1af1d8]
       mongod(__wt_turtle_read+0x30B) [0x55970c87ceab]
       mongod(__wt_metadata_search+0x23E) [0x55970c87b79e]
       mongod(__wt_conn_btree_open+0x70) [0x55970c831440]
       mongod(__wt_session_get_btree+0xF3) [0x55970c8b8c93]
       mongod(__wt_session_get_btree+0x6D9) [0x55970c8b9279]
       mongod(__wt_session_get_btree_ckpt+0x14C) [0x55970c8b950c]
       mongod(__wt_curfile_open+0xB8) [0x55970c840cd8]
       mongod(+0x1F0FC00) [0x55970c8afc00]
       mongod(__wt_metadata_cursor_open+0x6E) [0x55970c87adfe]
       mongod(__wt_metadata_cursor+0x4B) [0x55970c87aeeb]
       mongod(wiredtiger_open+0x1698) [0x55970c82dab8]
       mongod(_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0xAF9) [0x55970bbb2b19]
       mongod(+0x120B94C) [0x55970bbab94c]
       mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x6E0) [0x55970ba9c1b0]
       mongod(+0x7F066C) [0x55970b19066c]
       mongod(main+0x95F) [0x55970b1b07ff]
       libc.so.6(__libc_start_main+0xF5) [0x7f3ab03fbb45]
       mongod(+0x86C8EF) [0x55970b20c8ef]
      -----  END BACKTRACE  -----
      

      Attachments

        1. MongoLog
          123 kB
        2. repair_attempt.tar.gz
          8 kB
        3. WiredTiger_new.zip
          8 kB
        4. WiredTiger.zip
          8 kB

        Activity

          People

            kelsey.schubert@mongodb.com Kelsey T Schubert
            tadejcek Tadej Markun
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: