[SERVER-61850] mongodb crash with "Got signal: 11 (Segmentation fault)" Created: 02/Dec/21  Updated: 08/Mar/22  Resolved: 08/Mar/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.12
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: li bo Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

mongodb crash with "Got signal: 11 (Segmentation fault)"

db version v4.0.12 PSS

log:

2021-12-01T08:06:55.463+0800 I ACCESS   [conn15783385] Successfully authenticated as principal nagios on admin from client 127.0.0.1:49194
2021-12-01T08:06:55.474+0800 I NETWORK  [conn15783385] end connection 127.0.0.1:49194 (170 connections now open)
2021-12-01T08:06:56.435+0800 F -        [conn15779691] Invalid access at address: 0x562136602a33
2021-12-01T08:06:56.474+0800 F -        [conn15779691] Got signal: 11 (Segmentation fault).
 0x562151c1b771 0x562151c1a989 0x562151c1aff6 0x7fb03686e390 0x56215187e2b0 0x562151862bfb 0x562150b18ffb 0x562150b1a42c 0x5621508f4771 0x5621508ecb1d 0x5621508eac72 0x5621502b624c 0x5621502b7b29 0x5621502b8a61 0x5621502a428a 0x5621502b0e6a 0x5621502abb27 0x5621502af341 0x56215146cb52 0x5621502a9d10 0x5621502ace55 0x5621502ab267 0x5621502abbad 0x5621502af341 0x56215146d0b5 0x562151b730d4 0x7fb0368646ba 0x7fb03659a41d
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"56214F7E1000","o":"243A771","s":"_ZN5mongo15printStackTraceERSo"},{"b":"56214F7E1000","o":"2439989"},{"b":"56214F7E1000","o":"2439FF6"},{"b":"7FB03685D000","o":"11390"},{"b":"56214F7E1000","o":"209D2B0","s":"_ZN5mongo16UpdateObjectNode13parseAndMergeEPS0_NS_13modifiertable12ModifierTypeENS_11BSONElementERKN5boost13intrusive_ptrINS_17ExpressionContextEEERKSt3mapINS_10StringDataESt10unique_ptrINS_25ExpressionWithPlaceholderESt14default_deleteISE_EESt4lessISC_ESaISt4pairIKSC_SH_EEERSt3setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESI_ISX_ESaISX_EE"},{"b":"56214F7E1000","o":"2081BFB","s":"_ZN5mongo12UpdateDriver5parseERKNS_7BSONObjERKSt3mapINS_10StringDataESt10unique_ptrINS_25ExpressionWithPlaceholderESt14default_deleteIS7_EESt4lessIS5_ESaISt4pairIKS5_SA_EEEb"},{"b":"56214F7E1000","o":"1337FFB","s":"_ZN5mongo12ParsedUpdate11parseUpdateEv"},{"b":"56214F7E1000","o":"133942C","s":"_ZN5mongo12ParsedUpdate12parseRequestEv"},{"b":"56214F7E1000","o":"1113771","s":"_ZN5mongo14performUpdatesEPNS_16OperationContextERKNS_9write_ops6UpdateE"},{"b":"56214F7E1000","o":"110BB1D"},{"b":"56214F7E1000","o":"1109C72"},{"b":"56214F7E1000","o":"AD524C"},{"b":"56214F7E1000","o":"AD6B29"},{"b":"56214F7E1000","o":"AD7A61","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"56214F7E1000","o":"AC328A","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"56214F7E1000","o":"ACFE6A","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"56214F7E1000","o":"ACAB27","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56214F7E1000","o":"ACE341"},{"b":"56214F7E1000","o":"1C8BB52","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"56214F7E1000","o":"AC8D10","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"56214F7E1000","o":"ACBE55","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"56214F7E1000","o":"ACA267","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"56214F7E1000","o":"ACABAD","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56214F7E1000","o":"ACE341"},{"b":"56214F7E1000","o":"1C8C0B5"},{"b":"56214F7E1000","o":"23920D4"},{"b":"7FB03685D000","o":"76BA"},{"b":"7FB036493000","o":"10741D","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.0.12", "gitVersion" : "5776e3cbf9e7afe86e6b29e22520ffb6766e95d4", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.15.0-1047-gcp", "version" : "#50-Ubuntu SMP Wed Oct 2 00:50:34 UTC 2019", "machine" : "x86_64" }, "somap" : [ { "b" : "56214F7E1000", "elfType" : 3, "buildId" : "19988BA83768E2FC38C79673D456524C6C4275D6" }, { "b" : "7FFECC8C3000", "elfType" : 3, "buildId" : "61EC91AFCD8BD9A99B7B4BAF50928F945B334D36" }, { "b" : "7FB037C6E000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "93EA3FEB7AC8F28D7F4F59F5B94226A20A6F9023" }, { "b" : "7FB037A53000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "50A923F8DAFECBCD969C8573116A38C18D0E24D5" }, { "b" : "7FB03760E000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "15FFEB43278726B025F020862BF51302822A40EC" }, { "b" : "7FB0373A5000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "FF69EA60EBE05F2DD689D2B26FC85A73E5FBC3A0" }, { "b" : "7FB0371A1000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "37BFC3D8F7E3B022DAC7943B1A5FACD40CEBF0AD" }, { "b" : "7FB036F99000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "69143E8B39040C964D3958490535322675F15DD3" }, { "b" : "7FB036C90000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "BAD67A84E56E73D031AE507261DA066B35949D34" }, { "b" : "7FB036A7A000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7FB03685D000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "B17C21299099640A6D863E423D99265824E7BB16" }, { "b" : "7FB036493000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "1CA54A6E0D76188105B12E49FE6B8019BF08803A" }, { "b" : "7FB037EDD000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "C0ADBAD6F9A33944F2B3567C078EC472A1DAE98E" }, { "b" : "7FB036260000", "path" : "/usr/lib/x86_64-linux-gnu/libidn.so.11", "elfType" : 3, "buildId" : "E09D3783AD1D0BBCD3204FA01E4EF6D756E18F57" }, { "b" : "7FB036044000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "8D1CC1204D6B6D33BD1D2C5A2A0516A2234322CF" }, { "b" : "7FB035DFA000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "41971A4A3CCDC54A447F41DF4BD96C948C546E0E" }, { "b" : "7FB035BEB000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "5B146086EC173C299BA45F0C92E068D68CC7AD37" }, { "b" : "7FB03599A000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "D02A94DB90567C2C45F40C8565809A041FD22A55" }, { "b" : "7FB035780000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "8D9BD4CE26E45EF16075C67D5F5EEAFD8B562832" }, { "b" : "7FB035450000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "3CE893F6D1382C2C7648DCCB06E71B1C7E0861CC" }, { "b" : "7FB03521D000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "B11678F560199547DCF726384EA39153EE0DFABF" }, { "b" : "7FB034FE7000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "D6B36C5A463EE0FA84FDD6D5FD3F7726EDB90D54" }, { "b" : "7FB034D67000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "7B3533D5998D20EE1A1BE3F87789B69041E7F620" }, { "b" : "7FB034A95000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "0EEF7058B0737B68BDF89E5DC604D0AC389C8BB1" }, { "b" : "7FB034866000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "FFBA483A43D9EF73925AC116811890C037523DA1" }, { "b" : "7FB034662000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "1E16CB57F699E215A2A8D4EFEF90883BC749B12D" }, { "b" : "7FB034457000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "B789D8D4B4FC333405AB34387D9237F954060EA4" }, { "b" : "7FB03423C000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "87783DF8A1058CD150F8886CB36340384093C18F" }, { "b" : "7FB033FFB000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "1FE877BE52A424D0636AFD4D35BB330E41D6E0F3" }, { "b" : "7FB033D97000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "A0E2D03FF5CF65937F4425D4EFD4D655243809EB" }, { "b" : "7FB033B84000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "E07E186694852D8F69459C6AB28A53F8DA3CE3B6" }, { "b" : "7FB033980000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "3364D4BF2113C4E8D17EF533867ECC99A53413D6" }, { "b" : "7FB033777000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "73A8EADBC85860662B24850E71D4AFBE22C33359" }, { "b" : "7FB0334ED000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "59E742306A4EA2872E061ECCE92F35FADDA75357" }, { "b" : "7FB03324B000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "E5C159E415406AE79D21056D752BA949C408B5B1" }, { "b" : "7FB033018000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "7D15576E1F096614D360784E4A01A1F5FAF908C9" }, { "b" : "7FB032E02000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "481DB33C28D88E43DA6BED65E1A7599407D4D818" }, { "b" : "7FB032BFA000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "9D9C958F1F4894AFEF6AECD90D1C430EA29AC34F" }, { "b" : "7FB0329D1000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType" : 3, "buildId" : "57E25072866B2D30CF02EBE7AE623B84F96FA700" }, { "b" : "7FB0327C2000", "path" : "/usr/lib/x86_64-linux-gnu/libheimbase.so.1", "elfType" : 3, "buildId" : "F6F1B4E9F89B716C4A0BA5819BDFFAF4A13EFB91" }, { "b" : "7FB032577000", "path" : "/usr/lib/x86_64-linux-gnu/libhx509.so.5", "elfType" : 3, "buildId" : "C60082E3BB78D0D42868D9B359B89BF66CE5A1A7" }, { "b" : "7FB0322A3000", "path" : "/usr/lib/x86_64-linux-gnu/libsqlite3.so.0", "elfType" : 3, "buildId" : "C646E090F27A5CAEE45AC2BE4EFBC01E550BD27B" }, { "b" : "7FB03206B000", "path" : "/lib/x86_64-linux-gnu/libcrypt.so.1", "elfType" : 3, "buildId" : "FD61CA7A6D603E94E5EFD5C88D8810AE104FCF40" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x562151c1b771]
 mongod(+0x2439989) [0x562151c1a989]
 mongod(+0x2439FF6) [0x562151c1aff6]
 libpthread.so.0(+0x11390) [0x7fb03686e390]
 mongod(_ZN5mongo16UpdateObjectNode13parseAndMergeEPS0_NS_13modifiertable12ModifierTypeENS_11BSONElementERKN5boost13intrusive_ptrINS_17ExpressionContextEEERKSt3mapINS_10StringDataESt10unique_ptrINS_25ExpressionWithPlaceholderESt14default_deleteISE_EESt4lessISC_ESaISt4pairIKSC_SH_EEERSt3setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESI_ISX_ESaISX_EE+0x420) [0x56215187e2b0]
 mongod(_ZN5mongo12UpdateDriver5parseERKNS_7BSONObjERKSt3mapINS_10StringDataESt10unique_ptrINS_25ExpressionWithPlaceholderESt14default_deleteIS7_EESt4lessIS5_ESaISt4pairIKS5_SA_EEEb+0x51B) [0x562151862bfb]
 mongod(_ZN5mongo12ParsedUpdate11parseUpdateEv+0x6B) [0x562150b18ffb]
 mongod(_ZN5mongo12ParsedUpdate12parseRequestEv+0x1BC) [0x562150b1a42c]
 mongod(_ZN5mongo14performUpdatesEPNS_16OperationContextERKNS_9write_ops6UpdateE+0x691) [0x5621508f4771]
 mongod(+0x110BB1D) [0x5621508ecb1d]
 mongod(+0x1109C72) [0x5621508eac72]
 mongod(+0xAD524C) [0x5621502b624c]
 mongod(+0xAD6B29) [0x5621502b7b29]
 mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x3C1) [0x5621502b8a61]
 mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x5621502a428a]
 mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xBA) [0x5621502b0e6a]
 mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x5621502abb27]
 mongod(+0xACE341) [0x5621502af341]
 mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x56215146cb52]
 mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x150) [0x5621502a9d10]
 mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xB55) [0x5621502ace55]
 mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x357) [0x5621502ab267]
 mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x5621502abbad]
 mongod(+0xACE341) [0x5621502af341]
 mongod(+0x1C8C0B5) [0x56215146d0b5]
 mongod(+0x23920D4) [0x562151b730d4]
 libpthread.so.0(+0x76BA) [0x7fb0368646ba]
 libc.so.6(clone+0x6D) [0x7fb03659a41d]
-----  END BACKTRACE  -----

 



 Comments   
Comment by Eric Sedor [ 08/Mar/22 ]

Hi,

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Regards,
Eric

Comment by Eric Sedor [ 31/Jan/22 ]

Hi gz.libo@139.com, have you seen this issue again, and are you able to provide a tar or zip of the mongod.log files leading up to this crash and the $dbpath/diagnostic.data directory?

Comment by Eric Sedor [ 10/Dec/21 ]

Hi gz.libo@139.com,

Would you please archive (tar or zip) mongod.log files leading up to this crash and the $dbpath/diagnostic.data directory (the contents are described here) and upload them to this support uploader location?

Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Thanks,
Eric

Generated at Thu Feb 08 05:53:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.