Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-37565

mongod continuously restarting after upgrade to 4.0.2

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.0.6, 4.1.7
    • Affects Version/s: 4.0.2, 4.0.3
    • Component/s: Security
    • Labels:
      None
    • Fully Compatible
    • ALL
    • v4.0
    • Hide
      • start with mongod version 3.6.7
      • stop mongod
      • upgrade to 4.0.1/4.0.2/4.0.3
      • start mongod
      • 600-700 connections to the upgraded member in the first 2 minutes
      • server restarts

      This is a trace from the mongodb log, maybe it helps:

      {"backtrace":[{"b":"7FF94AF13000","o":"23B8EA1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"7FF94AF13000","o":"23B80B9"},{"b":"7FF94AF13000","o":"23B859D"},{"b":"7FF94966C000","o":"11390"},{"b":"7FF9492A2000","o":"35428","s":"gsignal"},{"b":"7FF9492A2000","o":"3702A","s":"abort"},{"b":"7FF94AF13000","o":"A04503","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"7FF94AF13000","o":"234D657","s":"_ZN5mongo24secure_allocator_details8allocateEmm"},{"b":"7FF94AF13000","o":"1DA8D9D","s":"_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE10_firstStepEPNS_16OperationContextENS_10StringDataE"},{"b":"7FF94AF13000","o":"1DA9BF8","s":"_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE8stepImplEPNS_16OperationContextENS_10StringDataE"},{"b":"7FF94AF13000","o":"1B51B40"},{"b":"7FF94AF13000","o":"1B537C0"},{"b":"7FF94AF13000","o":"1E2A3F9","s":"_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE"},{"b":"7FF94AF13000","o":"AAE3DF"},{"b":"7FF94AF13000","o":"AB060F"},{"b":"7FF94AF13000","o":"AB2979"},{"b":"7FF94AF13000","o":"AB38B1","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"7FF94AF13000","o":"AA0EFA","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"7FF94AF13000","o":"AABC6A","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AA6937","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AAA141"},{"b":"7FF94AF13000","o":"1C5D902","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"7FF94AF13000","o":"AA4B4F","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"7FF94AF13000","o":"AA7CE5","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"7FF94AF13000","o":"AA6077","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AA69BD","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AAA141"},{"b":"7FF94AF13000","o":"1C5DE65"},{"b":"7FF94AF13000","o":"2312724"},{"b":"7FF94966C000","o":"76BA"},{"b":"7FF9492A2000","o":"10741D","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.0.2", "gitVersion" : "fc1573ba18aee42f97a3bb13b67af7d837826b47", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.13.0-107-generic", "version" : "#154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "b" : "7FF94AF13000", "elfType" : 3, "buildId" : "AAF90A0B71749BBE7B9E3CF14359342D6DCFF400" }, { "b" : "7FFEABCBC000", "elfType" : 3, "buildId" : "C1463BCC3775232AE965C05F22AD5EDF8D26BEA5" }, { "b" : "7FF94AA7C000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "5D192460BE804CC9B9807E6D49834431CBF20E9A" }, { "b" : "7FF94A861000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "6EF73266978476EF9F2FD2CF31E57F4597CB74F8" }, { "b" : "7FF94A41D000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "250E875F74377DFC74DE48BF80CCB237BB4EFF1D" }, { "b" : "7FF94A1B4000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "513282AC7EB386E2C0133FD9E1B6B8A0F38B047D" }, { "b" : "7FF949FB0000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "8CC8D0D119B142D839800BFF71FB71E73AEA7BD4" }, { "b" : "7FF949DA8000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "89C34D7A182387D76D5CDA1F7718F5D58824DFB3" }, { "b" : "7FF949A9F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "DFB85DE42DAFFD09640C8FE377D572DE3E168920" }, { "b" : "7FF949889000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7FF94966C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "CE17E023542265FC11D9BC8F534BB4F070493D30" }, { "b" : "7FF9492A2000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B5381A457906D279073822A5CEB24C4BFEF94DDB" }, { "b" : "7FF94ACEB000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5D7B6259552275A3C17BD4C3FD05F5A6BF40CAA5" }, { "b" : "7FF94906F000", "path" : "/usr/lib/x86_64-linux-gnu/libidn.so.11", "elfType" : 3, "buildId" : "E09D3783AD1D0BBCD3204FA01E4EF6D756E18F57" }, { "b" : "7FF948E53000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "8D1CC1204D6B6D33BD1D2C5A2A0516A2234322CF" }, { "b" : "7FF948C09000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "DB5180B568097E2A4690A5B40D36BD134C893FEE" }, { "b" : "7FF9489FA000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "D3B183C41F02C7CD18F906AAFD19C69C850F1CEB" }, { "b" : "7FF9487A9000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "DA0EC53E16B3AC6BDC56EAEFE1BFECDDC395FB2E" }, { "b" : "7FF94858F000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "8D9BD4CE26E45EF16075C67D5F5EEAFD8B562832" }, { "b" : "7FF94825F000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "3CE893F6D1382C2C7648DCCB06E71B1C7E0861CC" }, { "b" : "7FF94802C000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "B11678F560199547DCF726384EA39153EE0DFABF" }, { "b" : "7FF947DF6000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "D6B36C5A463EE0FA84FDD6D5FD3F7726EDB90D54" }, { "b" : "7FF947B76000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "7B3533D5998D20EE1A1BE3F87789B69041E7F620" }, { "b" : "7FF9478A4000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "16E3DBC6D048145939BB43BBFD7954D27421B00F" }, { "b" : "7FF947675000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "AEB4C08FC47F86C475E9D3996DFE5E9B403ACEBF" }, { "b" : "7FF947471000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "1E16CB57F699E215A2A8D4EFEF90883BC749B12D" }, { "b" : "7FF947266000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "DF3219B89E86920E901BAC4A80AA60F2B6134588" }, { "b" : "7FF94704B000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "96BCC7EB28D81B1469EED6F24FC083CBD58577BC" }, { "b" : "7FF946E0A000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "1FE877BE52A424D0636AFD4D35BB330E41D6E0F3" }, { "b" : "7FF946BA6000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "A0E2D03FF5CF65937F4425D4EFD4D655243809EB" }, { "b" : "7FF946993000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "E07E186694852D8F69459C6AB28A53F8DA3CE3B6" }, { "b" : "7FF94678F000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "3364D4BF2113C4E8D17EF533867ECC99A53413D6" }, { "b" : "7FF946586000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "73A8EADBC85860662B24850E71D4AFBE22C33359" }, { "b" : "7FF9462FC000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "59E742306A4EA2872E061ECCE92F35FADDA75357" }, { "b" : "7FF94605A000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "E5C159E415406AE79D21056D752BA949C408B5B1" }, { "b" : "7FF945E27000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "7D15576E1F096614D360784E4A01A1F5FAF908C9" }, { "b" : "7FF945C11000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "481DB33C28D88E43DA6BED65E1A7599407D4D818" }, { "b" : "7FF945A09000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "9D9C958F1F4894AFEF6AECD90D1C430EA29AC34F" }, { "b" : "7FF9457E0000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType&...
      mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x7ff94d2cbea1]
       mongod(+0x23B80B9) [0x7ff94d2cb0b9]
       mongod(+0x23B859D) [0x7ff94d2cb59d]
       libpthread.so.0(+0x11390) [0x7ff94967d390]
       libc.so.6(gsignal+0x38) [0x7ff9492d7428]
       libc.so.6(abort+0x16A) [0x7ff9492d902a]
       mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x7ff94b917503]
       mongod(_ZN5mongo24secure_allocator_details8allocateEmm+0x737) [0x7ff94d260657]
       mongod(_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE10_firstStepEPNS_16OperationContextENS_10StringDataE+0xBDD) [0x7ff94ccbbd9d]
       mongod(_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE8stepImplEPNS_16OperationContextENS_10StringDataE+0x98) [0x7ff94ccbcbf8]
       mongod(+0x1B51B40) [0x7ff94ca64b40]
       mongod(+0x1B537C0) [0x7ff94ca667c0]
       mongod(_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE+0xD9) [0x7ff94cd3d3f9]
       mongod(+0xAAE3DF) [0x7ff94b9c13df]
       mongod(+0xAB060F) [0x7ff94b9c360f]
       mongod(+0xAB2979) [0x7ff94b9c5979]
       mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x3C1) [0x7ff94b9c68b1]
       mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x7ff94b9b3efa]
       mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xBA) [0x7ff94b9bec6a]
       mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x7ff94b9b9937]
       mongod(+0xAAA141) [0x7ff94b9bd141]
       mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x7ff94cb70902]
       mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x15F) [0x7ff94b9b7b4f]
       mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xAF5) [0x7ff94b9bace5]
       mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x357) [0x7ff94b9b9077]
       mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x7ff94b9b99bd]
       mongod(+0xAAA141) [0x7ff94b9bd141]
       mongod(+0x1C5DE65) [0x7ff94cb70e65]
       mongod(+0x2312724) [0x7ff94d225724]
       libpthread.so.0(+0x76BA) [0x7ff9496736ba]
       libc.so.6(clone+0x6D) [0x7ff9493a941d]
      -----  END BACKTRACE  -----
      
      Show
      start with mongod version 3.6.7 stop mongod upgrade to 4.0.1/4.0.2/4.0.3 start mongod 600-700 connections to the upgraded member in the first 2 minutes server restarts This is a trace from the mongodb log, maybe it helps: {"backtrace":[{"b":"7FF94AF13000","o":"23B8EA1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"7FF94AF13000","o":"23B80B9"},{"b":"7FF94AF13000","o":"23B859D"},{"b":"7FF94966C000","o":"11390"},{"b":"7FF9492A2000","o":"35428","s":"gsignal"},{"b":"7FF9492A2000","o":"3702A","s":"abort"},{"b":"7FF94AF13000","o":"A04503","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"7FF94AF13000","o":"234D657","s":"_ZN5mongo24secure_allocator_details8allocateEmm"},{"b":"7FF94AF13000","o":"1DA8D9D","s":"_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE10_firstStepEPNS_16OperationContextENS_10StringDataE"},{"b":"7FF94AF13000","o":"1DA9BF8","s":"_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE8stepImplEPNS_16OperationContextENS_10StringDataE"},{"b":"7FF94AF13000","o":"1B51B40"},{"b":"7FF94AF13000","o":"1B537C0"},{"b":"7FF94AF13000","o":"1E2A3F9","s":"_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE"},{"b":"7FF94AF13000","o":"AAE3DF"},{"b":"7FF94AF13000","o":"AB060F"},{"b":"7FF94AF13000","o":"AB2979"},{"b":"7FF94AF13000","o":"AB38B1","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"7FF94AF13000","o":"AA0EFA","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"7FF94AF13000","o":"AABC6A","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AA6937","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AAA141"},{"b":"7FF94AF13000","o":"1C5D902","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"7FF94AF13000","o":"AA4B4F","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"7FF94AF13000","o":"AA7CE5","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"7FF94AF13000","o":"AA6077","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AA69BD","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"7FF94AF13000","o":"AAA141"},{"b":"7FF94AF13000","o":"1C5DE65"},{"b":"7FF94AF13000","o":"2312724"},{"b":"7FF94966C000","o":"76BA"},{"b":"7FF9492A2000","o":"10741D","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.0.2", "gitVersion" : "fc1573ba18aee42f97a3bb13b67af7d837826b47", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.13.0-107-generic", "version" : "#154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "b" : "7FF94AF13000", "elfType" : 3, "buildId" : "AAF90A0B71749BBE7B9E3CF14359342D6DCFF400" }, { "b" : "7FFEABCBC000", "elfType" : 3, "buildId" : "C1463BCC3775232AE965C05F22AD5EDF8D26BEA5" }, { "b" : "7FF94AA7C000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "5D192460BE804CC9B9807E6D49834431CBF20E9A" }, { "b" : "7FF94A861000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "6EF73266978476EF9F2FD2CF31E57F4597CB74F8" }, { "b" : "7FF94A41D000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "250E875F74377DFC74DE48BF80CCB237BB4EFF1D" }, { "b" : "7FF94A1B4000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "513282AC7EB386E2C0133FD9E1B6B8A0F38B047D" }, { "b" : "7FF949FB0000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "8CC8D0D119B142D839800BFF71FB71E73AEA7BD4" }, { "b" : "7FF949DA8000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "89C34D7A182387D76D5CDA1F7718F5D58824DFB3" }, { "b" : "7FF949A9F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "DFB85DE42DAFFD09640C8FE377D572DE3E168920" }, { "b" : "7FF949889000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7FF94966C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "CE17E023542265FC11D9BC8F534BB4F070493D30" }, { "b" : "7FF9492A2000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B5381A457906D279073822A5CEB24C4BFEF94DDB" }, { "b" : "7FF94ACEB000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5D7B6259552275A3C17BD4C3FD05F5A6BF40CAA5" }, { "b" : "7FF94906F000", "path" : "/usr/lib/x86_64-linux-gnu/libidn.so.11", "elfType" : 3, "buildId" : "E09D3783AD1D0BBCD3204FA01E4EF6D756E18F57" }, { "b" : "7FF948E53000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "8D1CC1204D6B6D33BD1D2C5A2A0516A2234322CF" }, { "b" : "7FF948C09000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "DB5180B568097E2A4690A5B40D36BD134C893FEE" }, { "b" : "7FF9489FA000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "D3B183C41F02C7CD18F906AAFD19C69C850F1CEB" }, { "b" : "7FF9487A9000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "DA0EC53E16B3AC6BDC56EAEFE1BFECDDC395FB2E" }, { "b" : "7FF94858F000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "8D9BD4CE26E45EF16075C67D5F5EEAFD8B562832" }, { "b" : "7FF94825F000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "3CE893F6D1382C2C7648DCCB06E71B1C7E0861CC" }, { "b" : "7FF94802C000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "B11678F560199547DCF726384EA39153EE0DFABF" }, { "b" : "7FF947DF6000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "D6B36C5A463EE0FA84FDD6D5FD3F7726EDB90D54" }, { "b" : "7FF947B76000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "7B3533D5998D20EE1A1BE3F87789B69041E7F620" }, { "b" : "7FF9478A4000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "16E3DBC6D048145939BB43BBFD7954D27421B00F" }, { "b" : "7FF947675000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "AEB4C08FC47F86C475E9D3996DFE5E9B403ACEBF" }, { "b" : "7FF947471000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "1E16CB57F699E215A2A8D4EFEF90883BC749B12D" }, { "b" : "7FF947266000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "DF3219B89E86920E901BAC4A80AA60F2B6134588" }, { "b" : "7FF94704B000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "96BCC7EB28D81B1469EED6F24FC083CBD58577BC" }, { "b" : "7FF946E0A000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "1FE877BE52A424D0636AFD4D35BB330E41D6E0F3" }, { "b" : "7FF946BA6000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "A0E2D03FF5CF65937F4425D4EFD4D655243809EB" }, { "b" : "7FF946993000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "E07E186694852D8F69459C6AB28A53F8DA3CE3B6" }, { "b" : "7FF94678F000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "3364D4BF2113C4E8D17EF533867ECC99A53413D6" }, { "b" : "7FF946586000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "73A8EADBC85860662B24850E71D4AFBE22C33359" }, { "b" : "7FF9462FC000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "59E742306A4EA2872E061ECCE92F35FADDA75357" }, { "b" : "7FF94605A000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "E5C159E415406AE79D21056D752BA949C408B5B1" }, { "b" : "7FF945E27000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "7D15576E1F096614D360784E4A01A1F5FAF908C9" }, { "b" : "7FF945C11000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "481DB33C28D88E43DA6BED65E1A7599407D4D818" }, { "b" : "7FF945A09000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "9D9C958F1F4894AFEF6AECD90D1C430EA29AC34F" }, { "b" : "7FF9457E0000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType&... mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x7ff94d2cbea1] mongod(+0x23B80B9) [0x7ff94d2cb0b9] mongod(+0x23B859D) [0x7ff94d2cb59d] libpthread.so.0(+0x11390) [0x7ff94967d390] libc.so.6(gsignal+0x38) [0x7ff9492d7428] libc.so.6(abort+0x16A) [0x7ff9492d902a] mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x7ff94b917503] mongod(_ZN5mongo24secure_allocator_details8allocateEmm+0x737) [0x7ff94d260657] mongod(_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE10_firstStepEPNS_16OperationContextENS_10StringDataE+0xBDD) [0x7ff94ccbbd9d] mongod(_ZN5mongo24SaslSCRAMServerMechanismINS_15SCRAMSHA1PolicyEE8stepImplEPNS_16OperationContextENS_10StringDataE+0x98) [0x7ff94ccbcbf8] mongod(+0x1B51B40) [0x7ff94ca64b40] mongod(+0x1B537C0) [0x7ff94ca667c0] mongod(_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE+0xD9) [0x7ff94cd3d3f9] mongod(+0xAAE3DF) [0x7ff94b9c13df] mongod(+0xAB060F) [0x7ff94b9c360f] mongod(+0xAB2979) [0x7ff94b9c5979] mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x3C1) [0x7ff94b9c68b1] mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x7ff94b9b3efa] mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xBA) [0x7ff94b9bec6a] mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x7ff94b9b9937] mongod(+0xAAA141) [0x7ff94b9bd141] mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x7ff94cb70902] mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x15F) [0x7ff94b9b7b4f] mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xAF5) [0x7ff94b9bace5] mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x357) [0x7ff94b9b9077] mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x7ff94b9b99bd] mongod(+0xAAA141) [0x7ff94b9bd141] mongod(+0x1C5DE65) [0x7ff94cb70e65] mongod(+0x2312724) [0x7ff94d225724] libpthread.so.0(+0x76BA) [0x7ff9496736ba] libc.so.6(clone+0x6D) [0x7ff9493a941d] ----- END BACKTRACE -----
    • Security 2018-12-03, Security 2018-12-17, Security 2018-12-31

      Hello,

      I am trying to upgrade a replicaset from version 3.6.7 to 4.0.2. I started with one of the secondaries, everything was ok in the first 2 minutes after the upgrade but then the server keeps restarting every 2 or 3 minutes. All the members of this replicaset have a lot of connections (~1000) but after the server restarts it gets ~600-700 connections and then it crashes again. 

      I have upgraded a different replicaset which doesn't have this many connections and it worked fine so I think this is the issue here.

            Assignee:
            patrick.freed@mongodb.com Patrick Freed
            Reporter:
            andrada Andrada Nastasie
            Votes:
            1 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: