Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16214

fatal assertion on checkpoint

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Storage
    • Labels:
      None
    • ALL

      Got this on a workload rerun, deleting and starting with new DB directory:

      362E332B29242426353F242C21393A33363D3E2C21292C2A2D2331272C21242C3A2F2D373434313337332522372F32272E242334372A3831372F363834282F30263223...) } ] } keyUpdates:0  reslen:40 631ms
      2014-11-18T04:37:38.914-0600 I WRITES   [conn21] insert ycsb.usertable query: { _id: "user6210917365717213696", field0: BinData(0, 3E3F283F263B323B28342422392C312C3A2F27303534323F242320223D2C34
      2525313B3D302D2D29353F3B3E3E282F3D36212E3D3E282E25373D21383A3D352025252F3D3C21...) } ninserted:1 keyUpdates:0  789ms
      2014-11-18T04:37:38.914-0600 I QUERY    [conn21] command ycsb.$cmd command: insert { insert: "usertable", ordered: true, documents: [ { _id: "user6210917365717213696", field0: BinData(0, 3E3F28
      3F263B323B28342422392C312C3A2F27303534323F242320223D2C342525313B3D302D2D29353F3B3E3E282F3D36212E3D3E282E25373D21383A3D352025252F3D3C21...) } ] } keyUpdates:0  reslen:40 789ms
      2014-11-18T04:37:40.983-0600 E STORAGE  WiredTiger (22) [1416307060:983363][26124:0x7f637a46a700], file:WiredTiger.wt, session.checkpoint: live.alloc: merge range 4096-8192 overlaps with existi
      ng range 4096-8192: Invalid argument
      2014-11-18T04:37:40.983-0600 E STORAGE  WiredTiger (-31803) [1416307060:983490][26124:0x7f637a46a700], file:WiredTiger.wt, session.checkpoint: the process must exit and restart: WT_PANIC: Wired
      Tiger library panic
      2014-11-18T04:37:40.983-0600 I -        [conn20] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn16] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn17] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn14] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn10] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn11] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn12] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn22] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn6] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        Fatal Assertion 28558
      2014-11-18T04:37:40.983-0600 I -        [conn4] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn8] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn19] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn5] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn21] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn18] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn15] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn9] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn3] Fatal Assertion 28559
      2014-11-18T04:37:40.983-0600 I -        [conn7] Fatal Assertion 28559
      2014-11-18T04:37:40.994-0600 I CONTROL  [conn14]
       0xfd6f8e 0xf889af 0xf73050 0xdfc75e 0xdfc7f0 0xe0983d 0xe0714d 0xe07249 0xb39955 0xb399c9 0x808e7c 0xf9b531 0x31dac079d1 0x31da8e89dd
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"BD6F8E"},{"b":"400000","o":"B889AF"},{"b":"400000","o":"B73050"},{"b":"400000","o":"9FC75E"},{"b":"400000","o":"9FC7F0"},{"b":"400000","o":"A0983D"},{"b":"40000
      0","o":"A0714D"},{"b":"400000","o":"A07249"},{"b":"400000","o":"739955"},{"b":"400000","o":"7399C9"},{"b":"400000","o":"408E7C"},{"b":"400000","o":"B9B531"},{"b":"31DAC00000","o":"79D1"},{"b":"
      31DA800000","o":"E89DD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1-pre-", "gitVersion" : "7f96d80fa8fcad7fb59bebae7e83db1a9f0120ad", "uname" : { "sysname" : "Linux", "release" : "2.6.32-50
      4.el6.x86_64", "version" : "#1 SMP Wed Oct 15 04:27:16 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "AC5F74683E5B4906BF98D68F4F9B31D198F7A711" },
      { "b" : "7FFFA4EFF000", "elfType" : 3, "buildId" : "08E42C6C3D2CD1E5D68A43B717C9EB3D310F2DF0" }, { "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "B8DFF8E53D9F2B80C3C382E83EC17C8
      28B536A39" }, { "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "583411D8786F86A1D6B8741C502831E6122445A7" }, { "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "454F8FC6CC6502
      C6401E5F9E221564D80665D277" }, { "b" : "7F31A850D000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "F07F2E7CF4BFB393CC9BBE8CDC6463652E14DB07" }, { "path" : "/lib64/libm.so.
      6", "elfType" : 3, "buildId" : "7D8E9374F4A4EA38A7C1E763F32240EA113E4208" }, { "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "246C3BAB0AB093AFD59D34C8CBF29E786DE4BE97" }, { "path"
       : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "494ED3AE74E066D8F23DD697D3D95D4F8317ADBE" }, { "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "29EA72F5B09ADC9E955977F5AEF
      DA978C9AE4C03" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x2E) [0xfd6f8e]
       mongod(_ZN5mongo10logContextEPKc+0xDF) [0xf889af]
       mongod(_ZN5mongo13fassertFailedEi+0xC0) [0xf73050]
       mongod(_ZN5mongo12wtRCToStatusEiPKc+0x1CE) [0xdfc75e]
       mongod(_ZN5mongo13invariantWTOKEi+0x20) [0xdfc7f0]
       mongod(_ZN5mongo22WiredTigerSessionCache14releaseSessionEPNS_17WiredTigerSessionE+0x5D) [0xe0983d]
       mongod(_ZN5mongo22WiredTigerRecoveryUnitD1Ev+0x3D) [0xe0714d]
       mongod(_ZN5mongo22WiredTigerRecoveryUnitD0Ev+0x9) [0xe07249]
       mongod(_ZN5mongo20OperationContextImplD1Ev+0x45) [0xb39955]
       mongod(_ZN5mongo20OperationContextImplD0Ev+0x9) [0xb399c9]
       mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x27C) [0x808e7c]
       mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x491) [0xf9b531]
       libpthread.so.0(+0x79D1) [0x31dac079d1]
       libc.so.6(clone+0x6D) [0x31da8e89dd]
      -----  END BACKTRACE  -----
      

      This happened about 30 seconds into all insert workload.

            Assignee:
            Unassigned Unassigned
            Reporter:
            asya.kamsky@mongodb.com Asya Kamsky
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: