Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24659

The mongodb is crashed

    XMLWordPrintableJSON

Details

    • Icon: Question Question
    • Resolution: Done
    • Icon: Critical - P2 Critical - P2
    • None
    • None
    • Admin
    • None

    Description

      My mongodb which version is 3.0.7 is crashed, the following is detailed information:

      2016-06-20T12:43:50.886+0800 E STORAGE  [repl writer worker 1] WiredTiger (0) [1466397830:886662][12706:0x7fd756855700], file:collection-143-1451283047013864403.wt, cursor.search: read checksum error for 16384B block at offset 7593738240: calculated block checksum of 384598265 doesn't match expected checksum of 2923502613
      2016-06-20T12:43:50.886+0800 E STORAGE  [repl writer worker 1] WiredTiger (0) [1466397830:886748][12706:0x7fd756855700], file:collection-143-1451283047013864403.wt, cursor.search: collection-143-1451283047013864403.wt: encountered an illegal file format or internal value
      2016-06-20T12:43:50.886+0800 E STORAGE  [repl writer worker 1] WiredTiger (-31804) [1466397830:886769][12706:0x7fd756855700], file:collection-143-1451283047013864403.wt, cursor.search: the process must exit and restart: WT_PANIC: WiredTiger library panic
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 12] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 7] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 10] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 4] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 1] Fatal Assertion 28558
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 6] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 5] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 0] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 8] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 13] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 15] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 9] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 14] Fatal Assertion 28559
      2016-06-20T12:43:50.886+0800 I -        [repl writer worker 3] Fatal Assertion 28559
      2016-06-20T12:43:50.887+0800 I -        [repl writer worker 11] Fatal Assertion 28559
      2016-06-20T12:43:50.887+0800 I -        [repl writer worker 2] Fatal Assertion 28559
      2016-06-20T12:43:50.890+0800 I ACCESS   [conn108] Successfully authenticated as principal qxbdev on iEnterprise
      2016-06-20T12:43:50.891+0800 I NETWORK  [initandlisten] connection accepted from 10.47.90.78:54524 #110 (56 connections now open)
      2016-06-20T12:43:50.896+0800 I ACCESS   [conn107] Successfully authenticated as principal qxbdev on iEnterprise
      2016-06-20T12:43:50.898+0800 I NETWORK  [conn107] end connection 10.47.90.78:54520 (55 connections now open)
      2016-06-20T12:43:50.898+0800 I NETWORK  [initandlisten] connection accepted from 10.47.90.78:54525 #111 (56 connections now open)
      2016-06-20T12:43:50.898+0800 I CONTROL  [repl writer worker 6] 
       0xf5e979 0xf07861 0xeeb491 0xd886a0 0xd7d134 0xd7d192 0xd7d1d5 0xd6a485 0xa73f15 0x9f5710 0xa2273c 0xbc6464 0xbc6814 0xbc6e7d 0xb123c8 0xc33a78 0xc9bebf 0xc9ed65 0xefdd4b 0xfac904 0x7fd76ae8bdf3 0x7fd769f8e1ad
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"B5E979"},{"b":"400000","o":"B07861"},{"b":"400000","o":"AEB491"},{"b":"400000","o":"9886A0"},{"b":"400000","o":"97D134"},{"b":"400000","o":"97D192"},{"b":"400000","o":"97D1D5"},{"b":"400000","o":"96A485"},{"b":"400000","o":"673F15"},{"b":"400000","o":"5F5710"},{"b":"400000","o":"62273C"},{"b":"400000","o":"7C6464"},{"b":"400000","o":"7C6814"},{"b":"400000","o":"7C6E7D"},{"b":"400000","o":"7123C8"},{"b":"400000","o":"833A78"},{"b":"400000","o":"89BEBF"},{"b":"400000","o":"89ED65"},{"b":"400000","o":"AFDD4B"},{"b":"400000","o":"BAC904"},{"b":"7FD76AE84000","o":"7DF3"},{"b":"7FD769E98000","o":"F61AD"}],"processInfo":{ "mongodbVersion" : "3.0.7", "gitVersion" : "6ce7cbe8c6b899552dadd907604559806aa2e9bd", "uname" : { "sysname" : "Linux", "release" : "3.10.0-123.9.3.el7.x86_64", "version" : "#1 SMP Thu Nov 6 15:06:03 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFF190FE000", "elfType" : 3 }, { "b" : "7FD76AE84000", "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "b" : "7FD76AC7C000", "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "b" : "7FD76AA78000", "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7FD76A771000", "path" : "/lib64/libstdc++.so.6", "elfType" : 3 }, { "b" : "7FD76A46F000", "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "b" : "7FD76A259000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7FD769E98000", "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "b" : "7FD76B0A0000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5e979]
       mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf07861]
       mongod(_ZN5mongo13fassertFailedEi+0x61) [0xeeb491]
       mongod(_ZN5mongo17wtRCToStatus_slowEiPKc+0x2D0) [0xd886a0]
       mongod(_ZN5mongo22WiredTigerRecoveryUnit8_txnOpenEPNS_16OperationContextE+0xF4) [0xd7d134]
       mongod(_ZN5mongo22WiredTigerRecoveryUnit10getSessionEPNS_16OperationContextE+0x22) [0xd7d192]
       mongod(_ZN5mongo16WiredTigerCursorC1ERKSsmbPNS_16OperationContextE+0x35) [0xd7d1d5]
       mongod(_ZNK5mongo21WiredTigerIndexUnique9newCursorEPNS_16OperationContextEi+0x55) [0xd6a485]
       mongod(_ZNK5mongo22BtreeBasedAccessMethod10findSingleEPNS_16OperationContextERKNS_7BSONObjE+0x25) [0xa73f15]
       mongod(_ZN5mongo11IDHackStage4workEPm+0xD0) [0x9f5710]
       mongod(_ZN5mongo11UpdateStage4workEPm+0x7C) [0xa2273c]
       mongod(_ZN5mongo12PlanExecutor18getNextSnapshottedEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0xA4) [0xbc6464]
       mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x34) [0xbc6814]
       mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x3D) [0xbc6e7d]
       mongod(_ZN5mongo6updateEPNS_16OperationContextEPNS_8DatabaseERKNS_13UpdateRequestEPNS_7OpDebugE+0x118) [0xb123c8]
       mongod(_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEbb+0xF48) [0xc33a78]
       mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb+0x2EF) [0xc9bebf]
       mongod(_ZN5mongo4repl14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x65) [0xc9ed65]
       mongod(_ZN5mongo10threadpool6Worker4loopERKSs+0x2FB) [0xefdd4b]
       mongod(+0xBAC904) [0xfac904]
       libpthread.so.0(+0x7DF3) [0x7fd76ae8bdf3]
       libc.so.6(clone+0x6D) [0x7fd769f8e1ad]
      -----  END BACKTRACE  -----
      2016-06-20T12:43:50.899+0800 I -        [repl writer worker 6]
       
      ***aborting after fassert() failure
      
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            jin zhihao waynot
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: