Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17098

wiredTiger read checksum error

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Cannot Reproduce
    • Affects Version/s: 3.0.0-rc7
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
    • Environment:
      CentOS 6.5
    • Operating System:
      Linux
    • Steps To Reproduce:
      Hide

      Run sysbench against mongod on the same host (CentOS 6.5) in a single-node topology with Java driver ver 2.12.4.

      Show
      Run sysbench against mongod on the same host (CentOS 6.5) in a single-node topology with Java driver ver 2.12.4.

      Description

      With single-node sysbench workload running insert, mongod 3.0.0-rc7 from MCI encountered a wiredTiger library panic.

       118ms
      2015-01-28T19:27:45.207+0000 E STORAGE  WiredTiger (0) [1422473265:207337][21989:0x7f0ee4e0a700], file:WiredTiger.wt, session.checkpoint: read checksum error [4096B @ 36864, 2688513059 != 1
      97662958]
      2015-01-28T19:27:45.207+0000 E STORAGE  WiredTiger (0) [1422473265:207393][21989:0x7f0ee4e0a700], file:WiredTiger.wt, session.checkpoint: WiredTiger.wt: encountered an illegal file format o
      r internal value
      2015-01-28T19:27:45.207+0000 E STORAGE  WiredTiger (-31804) [1422473265:207407][21989:0x7f0ee4e0a700], file:WiredTiger.wt, session.checkpoint: the process must exit and restart: WT_PANIC: W
      iredTiger library panic
      2015-01-28T19:27:45.207+0000 I -        [conn4] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn6] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn9] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn5] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn2] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn3] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        [conn7] Fatal Assertion 28559
      2015-01-28T19:27:45.207+0000 I -        Fatal Assertion 28558
      2015-01-28T19:27:45.207+0000 I -        [conn8] Fatal Assertion 28559
      2015-01-28T19:27:45.217+0000 I CONTROL  [conn4]
       0xf3a9d9 0xee4b21 0xec8f41 0xd67729 0xd62b3a 0xd5bfd4 0xd58296 0x8f22de 0x8f2aac 0x993393 0x993f7a 0x994054 0x994755 0x99737d 0x9b71a4 0x9b80e3 0x9b8cdb 0xb87f95 0xa99f88 0x7e6730 0xef8aab 0x7f0eeede89d1 0x7f0eedf8986d
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"B3A9D9"},{"b":"400000","o":"AE4B21"},{"b":"400000","o":"AC8F41"},{"b":"400000","o":"967729"},{"b":"400000","o":"962B3A"},{"b":"400000","o":"95BFD4"},{"b":"400000","o":"958296"},{"b":"400000","o":"4F22DE"},{"b":"400000","o":"4F2AAC"},{"b":"400000","o":"593393"},{"b":"400000","o":"593F7A"},{"b":"400000","o":"594054"},{"b":"400000","o":"594755"},{"b":"400000","o":"59737D"},{"b":"400000","o":"5B71A4"},{"b":"400000","o":"5B80E3"},{"b":"400000","o":"5B8CDB"},{"b":"400000","o":"787F95"},{"b":"400000","o":"699F88"},{"b":"400000","o":"3E6730"},{"b":"400000","o":"AF8AAB"},{"b":"7F0EEEDE1000","o":"79D1"},{"b":"7F0EEDEA1000","o":"E886D"}],"processInfo":{ "mongodbVersion" : "3.0.0-rc7", "gitVersion" : "e4c60053b2967e16f765fa25d16aa6d629faa196", "uname" : { "sysname" : "Linux", "release" : "2.6.32-431.el6.x86_64", "version" : "#1 SMP Fri Nov 22 03:15:09 UTC 2013", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFFC1C5C000", "elfType" : 3 }, { "b" : "7F0EEEDE1000", "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "b" : "7F0EEEBD9000", "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "b" : "7F0EEE9D5000", "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7F0EEE6CF000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F0EEE44B000", "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "b" : "7F0EEE235000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F0EEDEA1000", "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "b" : "7F0EEEFFE000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf3a9d9]
       mongod(_ZN5mongo10logContextEPKc+0xE1) [0xee4b21]
       mongod(_ZN5mongo13fassertFailedEi+0x61) [0xec8f41]
       mongod(_ZN5mongo17wtRCToStatus_slowEiPKc+0x309) [0xd67729]
       mongod(_ZN5mongo17WiredTigerSession13releaseCursorEmP11__wt_cursor+0x1EA) [0xd62b3a]
       mongod(_ZN5mongo16WiredTigerCursorD1Ev+0x14) [0xd5bfd4]
       mongod(_ZN5mongo21WiredTigerRecordStore12insertRecordEPNS_16OperationContextEPKcib+0x126) [0xd58296]
       mongod(_ZN5mongo10Collection15_insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x4E) [0x8f22de]
       mongod(_ZN5mongo10Collection14insertDocumentEPNS_16OperationContextERKNS_7BSONObjEb+0x8C) [0x8f2aac]
       mongod(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0xA93) [0x993393]
       mongod(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x25A) [0x993f7a]
       mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0x34) [0x994054]
       mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x395) [0x994755]
       mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x15D) [0x99737d]
       mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x34) [0x9b71a4]
       mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xC13) [0x9b80e3]
       mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x28B) [0x9b8cdb]
       mongod(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_b+0x755) [0xb87f95]
       mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0xAC8) [0xa99f88]
       mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xE0) [0x7e6730]
       mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x32B) [0xef8aab]
       libpthread.so.0(+0x79D1) [0x7f0eeede89d1]
       libc.so.6(clone+0x6D) [0x7f0eedf8986d]
      -----  END BACKTRACE  -----
      2015-01-28T19:27:45.217+0000 I -        [conn4]
       
      ***aborting after fassert() failure

        Attachments

          Activity

            People

            Assignee:
            michael.cahill Michael Cahill
            Reporter:
            quentin.conner Quentin Conner
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: