Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38292

mongodb crash with Got signal: 11 (Segmentation fault)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: WiredTiger
    • Labels:
      None
    • Operating System:
      ALL
    • Sprint:
      Storage Engines 2019-02-25, Storage Engines 2019-06-03
    • Case:
    • Story Points:
      0

      Description

      We are also having this issue both on 3.2.19 and 3.4.14 (both are WT engine). Here's the stack from 3.4.14 logs. Can you brief me on what exactly is the fix in WT-4037. It is not clear in that ticket.

      2018-11-28T10:06:13.885-0600 F - [thread2] Invalid access at address: 0
      2018-11-28T10:06:13.906-0600 F - [thread2] Got signal: 11 (Segmentation fault). 0x562636fa4a51 0x562636fa3c69 0x562636fa42d6 0x7f74ee9647e0 0x56263716bca3 0x56263716bd7c 0x562637aa1d7a 0x5626378e0466 0x5626378e05ea 0x5626378e0c08 0x5626378e3ac3 0x562637934294 0x562637934e47 0x56263792f813 0x56263792fba7 0x5626379316a3 0x56263799b906 0x7f74ee95caa1 0x7f74ee6a9c4d
      ----- BEGIN BACKTRACE -----
      {"backtrace":[\{"b":"562635A2B000","o":"1579A51","s":"_ZN5mongo15printStackTraceERSo"},\{"b":"562635A2B000","o":"1578C69"},\{"b":"562635A2B000","o":"15792D6"},\{"b":"7F74EE955000","o":"F7E0"},\{"b":"562635A2B000","o":"1740CA3","s":"_ZN8tcmalloc11ThreadCache21ReleaseToCentralCacheEPNS0_8FreeListEmi"},\{"b":"562635A2B000","o":"1740D7C","s":"_ZN8tcmalloc11ThreadCache11ListTooLongEPNS0_8FreeListEm"},\{"b":"562635A2B000","o":"2076D7A","s":"_ZdlPvRKSt9nothrow_t"},\{"b":"562635A2B000","o":"1EB5466","s":"__wt_split_stash_discard"},\{"b":"562635A2B000","o":"1EB55EA"},\{"b":"562635A2B000","o":"1EB5C08"},\{"b":"562635A2B000","o":"1EB8AC3","s":"__wt_split_reverse"},\{"b":"562635A2B000","o":"1F09294"},\{"b":"562635A2B000","o":"1F09E47","s":"__wt_evict"},\{"b":"562635A2B000","o":"1F04813"},\{"b":"562635A2B000","o":"1F04BA7"},\{"b":"562635A2B000","o":"1F066A3","s":"__wt_evict_thread_run"},\{"b":"562635A2B000","o":"1F70906","s":"__wt_thread_run"},\{"b":"7F74EE955000","o":"7AA1"},\{"b":"7F74EE5C1000","o":"E8C4D","s":"clone"}],"processInfo":\{ "mongodbVersion" : "3.4.14", "gitVersion" : "fd954412dfc10e4d1e3e2dd4fac040f8b476b268", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "2.6.32-754.6.3.el6.x86_64", "version" : "#1 SMP Tue Oct 9 17:27:49 UTC 2018", "machine" : "x86_64" }, "somap" : [ \{ "b" : "562635A2B000", "elfType" : 3, "buildId" : "64CD384C41ACC8D81741F0DFC0F9A3D7756F81FF" }, \{ "b" : "7FFF3E8F6000", "elfType" : 3, "buildId" : "F9F48CC73D4D61AE273899B31855C6589EE5EA8D" }, \{ "b" : "7F74EF7FD000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "BECFB85A8BC084042D5BF2BA9E66325CE798B659" }, \{ "b" : "7F74EF418000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "CBDA444A7109874C5350AE9CEEF3F82F749B347F" }, \{ "b" : "7F74EF210000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "552CEC3216281CCFD7FA6432C723D50163255823" }, \{ "b" : "7F74EF00C000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "2AF795BFFD122309BA3359FEBABB5D0967403D17" }, \{ "b" : "7F74EED88000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "4AAEE970B045D8BF946578B9C7F3AB5CDE9AB44A" }, \{ "b" : "7F74EEB72000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "EDC925E58FE28DCA536993EB13179C739F1E6566" }, \{ "b" : "7F74EE955000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "4EA475CD3FD3B69B6C95D9381FA74B36DB4992EF" }, \{ "b" : "7F74EE5C1000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "BCA7789C2EA8E28CB7CE553E183AC7E7EE36F8A2" }, \{ "b" : "7F74EFA69000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "97AF4B77212F74CFF72B6C013E6AA2D74A97EF60" }, \{ "b" : "7F74EE37D000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "9A737F8BF10FC99C37CC404D3FC188F6E11FEDD9" }, \{ "b" : "7F74EE096000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "8D3D6E28DF6EB3752642A7031AAC17D39EA4265D" }, \{ "b" : "7F74EDE92000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "7EC54D6E88BB7D2C1284117C2A483496A01EAAF4" }, \{ "b" : "7F74EDC66000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "CC89B4C8CDCCD32BA610BC72784DC3B7E9BD9E19" }, \{ "b" : "7F74EDA50000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "5FA8E5038EC04A774AF72A9BB62DC86E1049C4D6" }, \{ "b" : "7F74ED845000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "E0C522C589F775C324330BE09CE67DC83950A213" }, \{ "b" : "7F74ED642000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "AF374BAFB7F5B139A0B431D3F06D82014AFF3251" }, \{ "b" : "7F74ED428000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "4786A2A5D30B121601958E84D643C70C13C4FBA5" }, \{ "b" : "7F74ED209000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B4576BE308DDCF7BC31F7304E4734C3D846D0236" } ] }}
       mongod-3.4(_ZN5mongo15printStackTraceERSo+0x41) [0x562636fa4a51]
       mongod-3.4(+0x1578C69) [0x562636fa3c69]
       mongod-3.4(+0x15792D6) [0x562636fa42d6]
       libpthread.so.0(+0xF7E0) [0x7f74ee9647e0]
       mongod-3.4(_ZN8tcmalloc11ThreadCache21ReleaseToCentralCacheEPNS0_8FreeListEmi+0xE3) [0x56263716bca3]
       mongod-3.4(_ZN8tcmalloc11ThreadCache11ListTooLongEPNS0_8FreeListEm+0x1C) [0x56263716bd7c]
       mongod-3.4(_ZdlPvRKSt9nothrow_t+0x26A) [0x562637aa1d7a]
       mongod-3.4(__wt_split_stash_discard+0xC6) [0x5626378e0466]
       mongod-3.4(+0x1EB55EA) [0x5626378e05ea]
       mongod-3.4(+0x1EB5C08) [0x5626378e0c08]
       mongod-3.4(__wt_split_reverse+0x83) [0x5626378e3ac3]
       mongod-3.4(+0x1F09294) [0x562637934294]
       mongod-3.4(__wt_evict+0xAC7) [0x562637934e47]
       mongod-3.4(+0x1F04813) [0x56263792f813]
       mongod-3.4(+0x1F04BA7) [0x56263792fba7]
       mongod-3.4(__wt_evict_thread_run+0xD3) [0x5626379316a3]
       mongod-3.4(__wt_thread_run+0x16) [0x56263799b906]
       libpthread.so.0(+0x7AA1) [0x7f74ee95caa1]
       libc.so.6(clone+0x6D) [0x7f74ee6a9c4d]
      ----- END BACKTRACE -----
      

       

      Please note that we are in the process of generating a core dump and will attach here when the mongod process crashes again.

        Attachments

        1. local_dmesg.txt
          225 kB
        2. local_system_log_Nov28.log
          9 kB

          Issue Links

            Activity

              People

              Assignee:
              alexander.gorrod Alexander Gorrod
              Reporter:
              manan@indeed.com Manan Shah
              Participants:
              Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: