[SERVER-31709] mongodb abnormal closing Created: 25/Oct/17 Updated: 22/Jan/18 Resolved: 25/Oct/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 3.4.6 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Minor - P4 |
| Reporter: | SuperSuJJ [X] | Assignee: | Mark Agarunov |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Participants: | |
| Case: | (copied to CRM) |
| Description |
|
os:cetnos 7.2 error log: ***aborting after fassert() failure 2017-10-25T12:06:26.183+0800 I ACCESS [conn572] Successfully authenticated as principal __system on local 0x7fa555f84a41 0x7fa555f83c59 0x7fa555f8413d 0x7fa553668370 0x7fa5532cd1d7 0x7fa5532ce8c8 0x7fa55522d7b3 0x7fa555d2ccad 0x7fa5553c9ab3 0x7fa5553cba45 0x7fa5553a314a 0x7fa5553bfb1d 0x7fa5553c1b9f 0x7fa55547f50f 0x7fa555480bf1 0x7fa555a98ef0 0x7fa55569ed68 0x7fa55529cd4d 0x7fa55529d67d 0x7fa555eec981 0x7fa553660dc5 0x7fa55338f76d , {"b":"7FA554A13000","o":"1570C59"}, {"b":"7FA554A13000","o":"157113D"}, {"b":"7FA553659000","o":"F370"}, {"b":"7FA553298000","o":"351D7","s":"gsignal"}, {"b":"7FA553298000","o":"368C8","s":"abort"}, {"b":"7FA554A13000","o":"81A7B3","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"}, {"b":"7FA554A13000","o":"1319CAD","s":"_ZN5mongo12SecureRandom6createEv"}, {"b":"7FA554A13000","o":"9B6AB3","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation10_firstStepERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EEPS7_"}, {"b":"7FA554A13000","o":"9B8A45","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"99014A","s":"_ZN5mongo31NativeSaslAuthenticationSession4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"9ACB1D"}, {"b":"7FA554A13000","o":"9AEB9F"}, {"b":"7FA554A13000","o":"A6C50F","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"A6DBF1","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"1085EF0","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"C8BD68","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"}, {"b":"7FA554A13000","o":"889D4D","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"}, {"b":"7FA554A13000","o":"88A67D"}, {"b":"7FA554A13000","o":"14D9981"}, {"b":"7FA553659000","o":"7DC5"}, {"b":"7FA553298000","o":"F776D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.4.6", "gitVersion" : "c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-327.el7.x86_64", "version" : "#1 SMP Thu Nov 19 22:10:57 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "b" : "7FA554A13000", "elfType" : 3, "buildId" : "60D7165C62663B8B732487CBB271FFEC7B04C8A6" }, { "b" : "7FFF9D1B4000", "elfType" : 3, "buildId" : "0F488C85309E4AB293D3B758748A3037B01F9885" }, { "b" : "7FA554583000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "90EAF65D9B0EEEB1424241281F7F197451D4317D" }, { "b" : "7FA554199000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "7278C69EE161D98DDD0FA00F92B67AD78C7B7F40" }, { "b" : "7FA553F91000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "82E77ADE22BC9FFF8D3458BD37331E7EDF174C28" }, { "b" : "7FA553D8D000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "C5F560504E1AF52E29679C3B52FF11121015D6BB" }, { "b" : "7FA553A8B000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "721C7CC9488EFA25F83B48AF713AB27DBE48EF3E" }, { "b" : "7FA553875000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "408B46E291B2D4C9612E27C0509D165D7E186D40" }, { "b" : "7FA553659000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "C3DEB1FA27CD0C1C3CC575B944ABACBA0698B0F2" }, { "b" : "7FA553298000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "2887288BE2E665C31B1C2DA3F1E8AE74631B25AA" }, { "b" : "7FA5547F1000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "8F3E366E2DB73C330A3791DEAE31AE9579099B44" }, { "b" : "7FA55304A000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "A2499C359AA179EE23324ED949C0E508E4434F10" }, { "b" : "7FA552D63000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "E09A34D9083DC6FEAF7018C09D55631DEEE2836D" }, { "b" : "7FA552B5F000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7FA55292D000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "BF8F00D7CB849ADB0B7A4703BC7B8D66AEE6A49C" }, { "b" : "7FA552717000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EA8E45DC8E395CC5E26890470112D97A1F1E0B65" }, { "b" : "7FA552508000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "1E7A92FDD6FB3871DA97F4BCA2E147E72B6B6E1F" }, { "b" : "7FA552304000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FA5520EA000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "FE7AE845A123A3DFC0FDC2408BCBC2BA8B61B158" }, { "b" : "7FA551EC3000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "76687CA31A406854DF3BCF8D03055656F56E6892" }, { "b" : "7FA551C62000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "30FA397B01197ECABC647CBD8E75FDD5B743D730" } ] }} 0x7fa555f84a41 0x7fa555f83c59 0x7fa555f8413d 0x7fa553668370 0x7fa5532cd1d7 0x7fa5532ce8c8 0x7fa55522d7b3 0x7fa555d2ccad 0x7fa5553c9ab3 0x7fa5553cba45 0x7fa5553a314a 0x7fa5553bfb1d 0x7fa5553c1b9f 0x7fa55547f50f 0x7fa555480bf1 0x7fa555a98ef0 0x7fa55569ed68 0x7fa55529cd4d 0x7fa55529d67d 0x7fa555eec981 0x7fa553660dc5 0x7fa55338f76d , {"b":"7FA554A13000","o":"1570C59"}, {"b":"7FA554A13000","o":"157113D"}, {"b":"7FA553659000","o":"F370"}, {"b":"7FA553298000","o":"351D7","s":"gsignal"}, {"b":"7FA553298000","o":"368C8","s":"abort"}, {"b":"7FA554A13000","o":"81A7B3","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"}, {"b":"7FA554A13000","o":"1319CAD","s":"_ZN5mongo12SecureRandom6createEv"}, {"b":"7FA554A13000","o":"9B6AB3","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation10_firstStepERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EEPS7_"}, {"b":"7FA554A13000","o":"9B8A45","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"99014A","s":"_ZN5mongo31NativeSaslAuthenticationSession4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"9ACB1D"}, {"b":"7FA554A13000","o":"9AEB9F"}, {"b":"7FA554A13000","o":"A6C50F","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"A6DBF1","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"1085EF0","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"C8BD68","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"}, {"b":"7FA554A13000","o":"889D4D","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"}, {"b":"7FA554A13000","o":"88A67D"}, {"b":"7FA554A13000","o":"14D9981"}, {"b":"7FA553659000","o":"7DC5"}, {"b":"7FA553298000","o":"F776D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.4.6", "gitVersion" : "c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-327.el7.x86_64", "version" : "#1 SMP Thu Nov 19 22:10:57 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "b" : "7FA554A13000", "elfType" : 3, "buildId" : "60D7165C62663B8B732487CBB271FFEC7B04C8A6" }, { "b" : "7FFF9D1B4000", "elfType" : 3, "buildId" : "0F488C85309E4AB293D3B758748A3037B01F9885" }, { "b" : "7FA554583000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "90EAF65D9B0EEEB1424241281F7F197451D4317D" }, { "b" : "7FA554199000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "7278C69EE161D98DDD0FA00F92B67AD78C7B7F40" }, { "b" : "7FA553F91000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "82E77ADE22BC9FFF8D3458BD37331E7EDF174C28" }, { "b" : "7FA553D8D000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "C5F560504E1AF52E29679C3B52FF11121015D6BB" }, { "b" : "7FA553A8B000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "721C7CC9488EFA25F83B48AF713AB27DBE48EF3E" }, { "b" : "7FA553875000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "408B46E291B2D4C9612E27C0509D165D7E186D40" }, { "b" : "7FA553659000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "C3DEB1FA27CD0C1C3CC575B944ABACBA0698B0F2" }, { "b" : "7FA553298000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "2887288BE2E665C31B1C2DA3F1E8AE74631B25AA" }, { "b" : "7FA5547F1000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "8F3E366E2DB73C330A3791DEAE31AE9579099B44" }, { "b" : "7FA55304A000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "A2499C359AA179EE23324ED949C0E508E4434F10" }, { "b" : "7FA552D63000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "E09A34D9083DC6FEAF7018C09D55631DEEE2836D" }, { "b" : "7FA552B5F000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7FA55292D000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "BF8F00D7CB849ADB0B7A4703BC7B8D66AEE6A49C" }, { "b" : "7FA552717000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EA8E45DC8E395CC5E26890470112D97A1F1E0B65" }, { "b" : "7FA552508000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "1E7A92FDD6FB3871DA97F4BCA2E147E72B6B6E1F" }, { "b" : "7FA552304000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FA5520EA000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "FE7AE845A123A3DFC0FDC2408BCBC2BA8B61B158" }, { "b" : "7FA551EC3000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "76687CA31A406854DF3BCF8D03055656F56E6892" }, { "b" : "7FA551C62000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "30FA397B01197ECABC647CBD8E75FDD5B743D730" } ] }} 0x7fa555f84a41 0x7fa555f83c59 0x7fa555f8413d 0x7fa553668370 0x7fa5532cd1d7 0x7fa5532ce8c8 0x7fa55522d7b3 0x7fa555d2ccad 0x7fa5553c9ab3 0x7fa5553cba45 0x7fa5553a314a 0x7fa5553bfb1d 0x7fa5553c1b9f 0x7fa55547f50f 0x7fa555480bf1 0x7fa555a98ef0 0x7fa55569ed68 0x7fa55529cd4d 0x7fa55529d67d 0x7fa555eec981 0x7fa553660dc5 0x7fa55338f76d , {"b":"7FA554A13000","o":"1570C59"}, {"b":"7FA554A13000","o":"157113D"}, {"b":"7FA553659000","o":"F370"}, {"b":"7FA553298000","o":"351D7","s":"gsignal"}, {"b":"7FA553298000","o":"368C8","s":"abort"}, {"b":"7FA554A13000","o":"81A7B3","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"}, {"b":"7FA554A13000","o":"1319CAD","s":"_ZN5mongo12SecureRandom6createEv"}, {"b":"7FA554A13000","o":"9B6AB3","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation10_firstStepERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EEPS7_"}, {"b":"7FA554A13000","o":"9B8A45","s":"_ZN5mongo31SaslSCRAMSHA1ServerConversation4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"99014A","s":"_ZN5mongo31NativeSaslAuthenticationSession4stepENS_10StringDataEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"}, {"b":"7FA554A13000","o":"9ACB1D"}, {"b":"7FA554A13000","o":"9AEB9F"}, {"b":"7FA554A13000","o":"A6C50F","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"A6DBF1","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"1085EF0","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"}, {"b":"7FA554A13000","o":"C8BD68","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"}, {"b":"7FA554A13000","o":"889D4D","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"}, {"b":"7FA554A13000","o":"88A67D"}, {"b":"7FA554A13000","o":"14D9981"}, {"b":"7FA553659000","o":"7DC5"}, {"b":"7FA553298000","o":"F776D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.4.6", "gitVersion" : "c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-327.el7.x86_64", "version" : "#1 SMP Thu Nov 19 22:10:57 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "b" : "7FA554A13000", "elfType" : 3, "buildId" : "60D7165C62663B8B732487CBB271FFEC7B04C8A6" }, { "b" : "7FFF9D1B4000", "elfType" : 3, "buildId" : "0F488C85309E4AB293D3B758748A3037B01F9885" }, { "b" : "7FA554583000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "90EAF65D9B0EEEB1424241281F7F197451D4317D" }, { "b" : "7FA554199000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "7278C69EE161D98DDD0FA00F92B67AD78C7B7F40" }, { "b" : "7FA553F91000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "82E77ADE22BC9FFF8D3458BD37331E7EDF174C28" }, { "b" : "7FA553D8D000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "C5F560504E1AF52E29679C3B52FF11121015D6BB" }, { "b" : "7FA553A8B000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "721C7CC9488EFA25F83B48AF713AB27DBE48EF3E" }, { "b" : "7FA553875000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "408B46E291B2D4C9612E27C0509D165D7E186D40" }, { "b" : "7FA553659000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "C3DEB1FA27CD0C1C3CC575B944ABACBA0698B0F2" }, { "b" : "7FA553298000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "2887288BE2E665C31B1C2DA3F1E8AE74631B25AA" }, { "b" : "7FA5547F1000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "8F3E366E2DB73C330A3791DEAE31AE9579099B44" }, { "b" : "7FA55304A000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "A2499C359AA179EE23324ED949C0E508E4434F10" }, { "b" : "7FA552D63000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "E09A34D9083DC6FEAF7018C09D55631DEEE2836D" }, { "b" : "7FA552B5F000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7FA55292D000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "BF8F00D7CB849ADB0B7A4703BC7B8D66AEE6A49C" }, { "b" : "7FA552717000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EA8E45DC8E395CC5E26890470112D97A1F1E0B65" }, { "b" : "7FA552508000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "1E7A92FDD6FB3871DA97F4BCA2E147E72B6B6E1F" }, { "b" : "7FA552304000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FA5520EA000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "FE7AE845A123A3DFC0FDC2408BCBC2BA8B61B158" }, { "b" : "7FA551EC3000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "76687CA31A406854DF3BCF8D03055656F56E6892" }, { "b" : "7FA551C62000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "30FA397B01197ECABC647CBD8E75FDD5B743D730" } ] }} |
| Comments |
| Comment by Mark Agarunov [ 25/Oct/17 ] | |
|
Hello Su, Thank you for the report. Looking over the error logs provided, it seems this behavior is due to hitting the limit of open files:
to increase the maximum number of files, increase the ulimit for mongod. Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-user group. Thanks, |