Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-14090

SIGABRT due to Invalid Free in tcmalloc causing memory corruption in 6.0.19

    • Type: Icon: Bug Bug
    • Resolution: Won't Do
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Cache and Eviction
    • Storage Engines
    • StorEng - Defined Pipeline

      The server encountered a fatal error resulting in an unexpected SIGABRT.

      The error log indicates that an invalid free operation was detected by tcmalloc, which led to memory corruption and the subsequent abort of the process. This issue appears to be directly related to memory management issue, specifically an attempt to free memory that was either already freed or otherwise invalid.

      12802:{"t":{"$date":"2025-02-01T03:48:40.770+00:00"},"s":"F",  "c":"CONTROL",  "id":6384300, "ctx":"thread15571","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
      12803:{"t":{"$date":"2025-02-01T03:48:40.951+00:00"},"s":"E",  "c":"CONTROL",  "id":31430,   "ctx":"thread15571","msg":"Error collecting stack trace","attr":{"error":"unw_get_proc_name(FFFFAF570810): unspecified (general) error\nunw_get_proc_name(FFFFAF570810): unspecified (general) error\n"}}
      12804:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31380,   "ctx":"thread15571","msg":"BACKTRACE","attr":{"bt":{"backtrace":[{"a":"AAAAEF403344","b":"AAAAEAA60000","o":"49A3344","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_117getStackTraceImplERKNS1_7OptionsE.constprop.206","C":"mongo::stack_trace_detail::(anonymous namespace)::getStackTraceImpl(mongo::stack_trace_detail::(anonymous namespace)::Options const&) [clone .constprop.206]","s+":"19C"},{"a":"AAAAEF404E70","b":"AAAAEAA60000","o":"49A4E70","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"48"},{"a":"AAAAEF3FF31C","b":"AAAAEAA60000","o":"499F31C","s":"abruptQuit","s+":"7C"},{"a":"FFFFAF570810","b":"FFFFAF570000","o":"810","s":"__kernel_rt_sigreturn","s+":"0"},{"a":"FFFFAE7DC834","b":"FFFFAE7AA000","o":"32834","s":"gsignal","s+":"B4"},{"a":"FFFFAE7DE140","b":"FFFFAE7AA000","o":"34140","s":"abort","s+":"160"},{"a":"AAAAEF54D0F8","b":"AAAAEAA60000","o":"4AED0F8","s":"_ZN8tcmalloc3LogENS_7LogModeEPKciNS_7LogItemES3_S3_S3_","C":"tcmalloc::Log(tcmalloc::LogMode, char const*, int, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem)","s+":"240"},{"a":"AAAAEF557380","b":"AAAAEAA60000","o":"4AF7380","s":"_ZN12_GLOBAL__N_111InvalidFreeEPv","C":"(anonymous namespace)::InvalidFree(void*)","s+":"40"},{"a":"AAAAED68DE38","b":"AAAAEAA60000","o":"2C2DE38","s":"__stash_discard","s+":"F8"},{"a":"AAAAED68E440","b":"AAAAEAA60000","o":"2C2E440","s":"__wt_stash_discard","s+":"50"},{"a":"AAAAED685480","b":"AAAAEAA60000","o":"2C25480","s":"__wt_session_release_resources","s+":"50"},{"a":"AAAAED633A18","b":"AAAAEAA60000","o":"2BD3A18","s":"__evict_lru_pages","s+":"F8"},{"a":"AAAAED637DF4","b":"AAAAEAA60000","o":"2BD7DF4","s":"__wt_evict_thread_run","s+":"6C"},{"a":"AAAAED69D550","b":"AAAAEAA60000","o":"2C3D550","s":"__thread_run","s+":"50"},{"a":"FFFFAE937230","b":"FFFFAE930000","o":"7230","s":"start_thread","s+":"B0"},{"a":"FFFFAE8857DC","b":"FFFFAE7AA000","o":"DB7DC","s":"thread_start","s+":"C"}],"processInfo":{"mongodbVersion":"6.0.19","gitVersion":"a7ada5ff3a4d8a1e2ed88f86bd6b3d1d16cb43c6","compiledModules":["enterprise"],"uname":{"sysname":"Linux","release":"5.15.173-118.169.2.mdb.amzn2.aarch64","version":"#1 SMP Fri Dec 20 06:37:48 UTC 2024","machine":"aarch64"},"somap":[{"b":"AAAAEAA60000","elfType":3,"buildId":"C0EEA18E3AA6AB0AE4BBFB375FFE33C5119F0D71"},{"b":"FFFFAF570000","path":"linux-vdso.so.1","elfType":3,"buildId":"BC7A96CAA99DCD9207FD79975D1D493572B599AE"},{"b":"FFFFAE930000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"635E47A0FDC7C1119AF7DAA8A199ED4E5A738518"},{"b":"FFFFAE7AA000","path":"/lib64/libc.so.6","elfType":3,"buildId":"E98BBED0281633200AD1F58EFEBAF59810BCF664"}]}}},"tags":[]}
      12805:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAEF403344","b":"AAAAEAA60000","o":"49A3344","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_117getStackTraceImplERKNS1_7OptionsE.constprop.206","C":"mongo::stack_trace_detail::(anonymous namespace)::getStackTraceImpl(mongo::stack_trace_detail::(anonymous namespace)::Options const&) [clone .constprop.206]","s+":"19C"}}}
      12806:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAEF404E70","b":"AAAAEAA60000","o":"49A4E70","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"48"}}}
      12807:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAEF3FF31C","b":"AAAAEAA60000","o":"499F31C","s":"abruptQuit","s+":"7C"}}}
      12808:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"FFFFAF570810","b":"FFFFAF570000","o":"810","s":"__kernel_rt_sigreturn","s+":"0"}}}
      12809:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"FFFFAE7DC834","b":"FFFFAE7AA000","o":"32834","s":"gsignal","s+":"B4"}}}
      12810:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"FFFFAE7DE140","b":"FFFFAE7AA000","o":"34140","s":"abort","s+":"160"}}}
      12811:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAEF54D0F8","b":"AAAAEAA60000","o":"4AED0F8","s":"_ZN8tcmalloc3LogENS_7LogModeEPKciNS_7LogItemES3_S3_S3_","C":"tcmalloc::Log(tcmalloc::LogMode, char const*, int, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem)","s+":"240"}}}
      12812:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAEF557380","b":"AAAAEAA60000","o":"4AF7380","s":"_ZN12_GLOBAL__N_111InvalidFreeEPv","C":"(anonymous namespace)::InvalidFree(void*)","s+":"40"}}}
      12813:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED68DE38","b":"AAAAEAA60000","o":"2C2DE38","s":"__stash_discard","s+":"F8"}}}
      12814:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED68E440","b":"AAAAEAA60000","o":"2C2E440","s":"__wt_stash_discard","s+":"50"}}}
      12815:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED685480","b":"AAAAEAA60000","o":"2C25480","s":"__wt_session_release_resources","s+":"50"}}}
      12816:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED633A18","b":"AAAAEAA60000","o":"2BD3A18","s":"__evict_lru_pages","s+":"F8"}}}
      12817:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED637DF4","b":"AAAAEAA60000","o":"2BD7DF4","s":"__wt_evict_thread_run","s+":"6C"}}}
      12818:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"AAAAED69D550","b":"AAAAEAA60000","o":"2C3D550","s":"__thread_run","s+":"50"}}}
      12819:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"FFFFAE937230","b":"FFFFAE930000","o":"7230","s":"start_thread","s+":"B0"}}}
      12820:{"t":{"$date":"2025-02-01T03:48:40.952+00:00"},"s":"I",  "c":"CONTROL",  "id":31445,   "ctx":"thread15571","msg":"Frame","attr":{"frame":{"a":"FFFFAE8857DC","b":"FFFFAE7AA000","o":"DB7DC","s":"thread_start","s+":"C"}}}
      

      MongoDB Version: 6.0.19 (Enterprise)

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            siddhartha.mahajan@mongodb.com Sid Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: