Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-49319

WiredTiger crash after shutdown

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • ALL

      Hello! I have issue with starting mongodb 4 after sudder shutdown of server.

      I have have only one large singe node mongodb without backup.
      I'm not sure if it correct to post it here

       Appreciate any help.

      mongod --repair

      log:

       

      2020-07-05T14:55:04.120+0000 E STORAGE [initandlisten] WiredTiger error (0) [1593960904:120370][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_bm_corrupt_dump, 135: {24331131842560, 188416, 969740655}: (chunk 184 of 184): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Raw: [1593960904:120370][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_bm_corrupt_dump, 135: {24331131842560, 188416, 969740655}: (chunk 184 of 184): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 2020-07-05T14:55:04.120+0000 E STORAGE [initandlisten] WiredTiger error (-31802) [1593960904:120454][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_block_read_off, 281: collection-5-7105085878558723654.wt: fatal read error: WT_ERROR: non-specific WiredTiger error Raw: [1593960904:120454][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_block_read_off, 281: collection-5-7105085878558723654.wt: fatal read error: WT_ERROR: non-specific WiredTiger error
 2020-07-05T14:55:04.120+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1593960904:120480][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1593960904:120480][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, WT_CURSOR.insert: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic
 2020-07-05T14:55:04.120+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1593960904:120504][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, txn-recover: __txn_op_apply, 287: operation apply failed during recovery: operation type 4 at LSN 268058/40238976: WT_PANIC: WiredTiger library panic Raw: [1593960904:120504][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, txn-recover: __txn_op_apply, 287: operation apply failed during recovery: operation type 4 at LSN 268058/40238976: WT_PANIC: WiredTiger library panic
 2020-07-05T14:55:04.120+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1593960904:120551][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, txn-recover: __wt_txn_recover, 706: Recovery failed: WT_PANIC: WiredTiger library panic Raw: [1593960904:120551][1:0x7f9d11fc6a80], file:collection-5-7105085878558723654.wt, txn-recover: __wt_txn_recover, 706: Recovery failed: WT_PANIC: WiredTiger library panic
 2020-07-05T14:55:04.121+0000 E STORAGE [initandlisten] WiredTiger error (0) [1593960904:121027][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 350: cache server: exiting with 18 pages in memory and 9 pages evicted Raw: [1593960904:121027][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 350: cache server: exiting with 18 pages in memory and 9 pages evicted
 2020-07-05T14:55:04.121+0000 E STORAGE [initandlisten] WiredTiger error (0) [1593960904:121066][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 355: cache server: exiting with 79216 image bytes in memory Raw: [1593960904:121066][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 355: cache server: exiting with 79216 image bytes in memory
 2020-07-05T14:55:04.121+0000 E STORAGE [initandlisten] WiredTiger error (0) [1593960904:121085][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 358: cache server: exiting with 131776 bytes in memory Raw: [1593960904:121085][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 358: cache server: exiting with 131776 bytes in memory
 2020-07-05T14:55:04.121+0000 E STORAGE [initandlisten] WiredTiger error (0) [1593960904:121100][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 364: cache server: exiting with 83777 bytes dirty and 4 pages dirty Raw: [1593960904:121100][1:0x7f9d11fc6a80], connection: __wt_cache_destroy, 364: cache server: exiting with 83777 bytes dirty and 4 pages dirty
 2020-07-05T14:55:04.122+0000 F STORAGE [initandlisten] Failed to salvage WiredTiger metadata: -31809: WT_TRY_SALVAGE: database corruption detected
 2020-07-05T14:55:04.122+0000 F - [initandlisten] Fatal Assertion 50947 at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 722
 2020-07-05T14:55:04.122+0000 F - [initandlisten]
***aborting after fassert() failure
 

       

        1. image-2020-07-10-14-43-27-233.png
          72 kB
          Anton Zolotukhin
        2. image-2020-07-10-14-45-44-647.png
          216 kB
          Anton Zolotukhin
        3. image-2020-07-10-14-46-57-804.png
          187 kB
          Anton Zolotukhin
        4. repair-1.log
          4.65 MB
          Anton Zolotukhin
        5. WiredTiger
          0.0 kB
          Anton Zolotukhin
        6. WiredTiger.turtle
          1.0 kB
          Anton Zolotukhin
        7. WiredTiger.wt
          164 kB
          Anton Zolotukhin

            Assignee:
            dmitry.agranat@mongodb.com Dmitry Agranat
            Reporter:
            ginn.tar.gz@gmail.com Anton Zolotukhin
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: