Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44489

When i add index on primary the SECONDARY crashed

    • Type: Icon: Question Question
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.0.2
    • Component/s: None
    • Labels:
      None

      Hi, Mongodb Teams !

      When I add some index on the primary node shard, like below:

      db.mycollection.createIndex({"colunm1":1,"colunm2":1,"colunm3":1},{background: true})
      

      when primary excuted completed,and SECONDARY excuted completed too ,  then SECONDARY carsh , I collect the log, what the problem caused the crashed ?

      Laster I start the SECONDARY is Ok .

      2019-11-07T09:41:37.738+0800 F - [conn26083] Invariant failure params.descriptor src/mongo/db/query/stage_builder.cpp 100
      2019-11-07T09:41:37.738+0800 F - [conn26083]
      ***aborting after invariant() failure
      2019-11-07T09:41:37.767+0800 F - [conn26082] Invariant failure params.descriptor src/mongo/db/query/stage_builder.cpp 100
      2019-11-07T09:41:37.767+0800 F - [conn26082]
      ***aborting after invariant() failure
      2019-11-07T09:41:37.874+0800 F - [conn26083] Got signal: 6 (Aborted).
       0x55a7dc251fe1 0x55a7dc2511f9 0x55a7dc2516dd 0x7f9bacfa85d0 0x7f9bacc02207 0x7f9bacc038f8 0x55a7da89391a 0x55a7db0d2875 0x55a7db0d1357 0x55a7db0d084f 0x55a7db0d161f 0x55a7db0d31c7 0x55a7db03ebf2 0x55a7db040387
       0x55a7db0c5cc2 0x55a7db0c9544 0x55a7db0ca5f4 0x55a7db0bf117 0x55a7db0bf6c6 0x55a7db0bff15 0x55a7dae8d42a 0x55a7dbcc0e99 0x55a7da93e0cf 0x55a7da940312 0x55a7da942689 0x55a7da9435d1 0x55a7da930bba 0x55a7da93b95a
       0x55a7da936617 0x55a7da939e21 0x55a7dbaf3432 0x55a7da93482f 0x55a7da9379c5 0x55a7da935d57 0x55a7da93669d 0x55a7da939e21 0x55a7dbaf3995 0x55a7dc1aafb4 0x7f9bacfa0dd5 0x7f9baccc9ead
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"55A7D9E8F000","o":"23C2FE1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55A7D9E8F000","o":"23C21F9"},{"b":"55A7D9E8F000","o":"23C26DD"},{"b":"7F9BACF99000","o":"F5D0"},{"b":"7F9BACBCC000","o
      ":"36207","s":"gsignal"},{"b":"7F9BACBCC000","o":"378F8","s":"abort"},{"b":"55A7D9E8F000","o":"A0491A","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j"},{"b"
      :"55A7D9E8F000","o":"1243875","s":"_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE"},{"b":"55A7D9E8F000","o":
      "1242357","s":"_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE"},{"b":"55A7D9E8F000","o":"124184F","s":"_ZN5m
      ongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE"},{"b":"55A7D9E8F000","o":"124261F","s":"_ZN5mongo11buildStagesEPN
      S_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE"},{"b":"55A7D9E8F000","o":"12441C7","s":"_ZN5mongo12StageBuilder5buildEPNS_16Operation
      ContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPNS_10WorkingSetEPPNS_9PlanStageE"},{"b":"55A7D9E8F000","o":"11AFBF2","s":"_ZN5mongo15CachedPlanStage6replanEPNS_15PlanYieldPolicyEb"},{"b":"5
      5A7D9E8F000","o":"11B1387","s":"_ZN5mongo15CachedPlanStage12pickBestPlanEPNS_15PlanYieldPolicyE"},{"b":"55A7D9E8F000","o":"1236CC2","s":"_ZN5mongo12PlanExecutor12pickBestPlanEPKNS_10CollectionE"},{"b":"55A7D9E8
      F000","o":"123A544","s":"_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextESt10unique_ptrINS_10WorkingSetESt14default_deleteIS4_EES3_INS_9PlanStageES5_IS8_EES3_INS_13QuerySolutionES5_ISB_EES3_INS_14CanonicalQ
      ueryES5_ISE_EEPKNS_10CollectionENS_15NamespaceStringENS0_11YieldPolicyE"},{"b":"55A7D9E8F000","o":"123B5F4","s":"_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextESt10unique_ptrINS_10WorkingSetESt14default_de
      leteIS4_EES3_INS_9PlanStageES5_IS8_EES3_INS_13QuerySolutionES5_ISB_EES3_INS_14CanonicalQueryES5_ISE_EEPKNS_10CollectionENS0_11YieldPolicyE"},{"b":"55A7D9E8F000","o":"1230117","s":"_ZN5mongo11getExecutorEPNS_16O
      perationContextEPNS_10CollectionESt10unique_ptrINS_14CanonicalQueryESt14default_deleteIS5_EENS_12PlanExecutor11YieldPolicyEm"},{"b":"55A7D9E8F000","o":"12306C6"},{"b":"55A7D9E8F000","o":"1230F15","s":"_ZN5mongo
      15getExecutorFindEPNS_16OperationContextEPNS_10CollectionERKNS_15NamespaceStringESt10unique_ptrINS_14CanonicalQueryESt14default_deleteIS8_EEm"},{"b":"55A7D9E8F000","o":"FFE42A"},{"b":"55A7D9E8F000","o":"1E31E99
      ","s":"_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE"},{"b":"55A7D9E8F000","o":"AAF0CF"},{"b":"55A7D9E8F000","o":"AB1312"},{"b":"55A7D9E8F000","o":"AB3689"},{"b":"55A
      7D9E8F000","o":"AB45D1","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"55A7D9E8F000","o":"AA1BBA","s":"_ZN5mongo23ServiceEntryPointMongod13hand
      leRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"55A7D9E8F000","o":"AAC95A","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"55A7D9E8F000","o":"AA7617","s":"_ZN5mongo19Serv
      iceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"55A7D9E8F000","o":"AAAE21"},{"b":"55A7D9E8F000","o":"1C64432","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15Ser
      viceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"55A7D9E8F000","o":"AA582F","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13Schedu
      leFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"55A7D9E8F000","o":"AA89C5","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"55A7D9E8F000","o":"AA6D57","s":"_ZN5mongo19Servi
      ceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"55A7D9E8F000","o":"AA769D","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"55A7D9E8F000","o":"AAAE21"},{"b":"55A7D9E8F00
      0","o":"1C64995"},{"b":"55A7D9E8F000","o":"231BFB4"},{"b":"7F9BACF99000","o":"7DD5"},{"b":"7F9BACBCC000","o":"FDEAD","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.0.2", "gitVersion" : "fc1573ba18aee42f97a
      3bb13b67af7d837826b47", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-957.12.2.el7.x86_64", "version" : "#1 SMP Tue May 14 21:24:32 UTC 2019", "machine" : "x86_64" }, "somap" : [ 
      { "b" : "55A7D9E8F000", "elfType" : 3, "buildId" : "86DF86886A5B030E7D7D4639DB20F4B807A3DAA4" }, { "b" : "7FFF83EAC000", "elfType" : 3, "buildId" : "D149FABD4DF969958863221F946E26C130432B47" }, { "b" : "7F9BAE3
      C6000", "path" : "/lib64/libcurl.so.4", "elfType" : 3, "buildId" : "9114859D3C4BEC47A03CA321EE367DCA799638CD" }, { "b" : "7F9BAE1AD000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "C444AE61E7C
      BB716FD9C18A0B46A7FE8F4FCF3E5" }, { "b" : "7F9BADD4B000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "3593FA778645A59EA272DBBB59D318C60940E792" }, { "b" : "7F9BADAD9000", "path" : "/lib64/lib
      ssl.so.10", "elfType" : 3, "buildId" : "AEF5E6F2240B55F90E9DF76CFBB8B9D9F5286583" }, { "b" : "7F9BAD8D5000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "357693C8F1F49D93010C4E31529C07CDD2BD3D08" }
      , { "b" : "7F9BAD6CD000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "EFDE2029C9A4A20BE5B8D8AE7E6551FF9B5755D2" }, { "b" : "7F9BAD3CB000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "
      5B14BE4D749631673523A61074C10959D50F5455" }, { "b" : "7F9BAD1B5000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "179F202998E429AA1215907F6D4C5C1BB9C90136" }, { "b" : "7F9BACF99000", "path" : "/
      lib64/libpthread.so.0", "elfType" : 3, "buildId" : "96900CB0FF25B26F2BBDF247DE1408242E4773D8" }, { "b" : "7F9BACBCC000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "EB9F22A3891E5FD3494DFD9ED199E20A
      E71BB08D" }, { "b" : "7F9BAE62F000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "A527FE72908703C5972AE384E78D1850D1881EE7" }, { "b" : "7F9BAC999000", "path" : "/lib64/libidn.so.11", "elf
      Type" : 3, "buildId" : "2B77BBEFFF65E94F3E0B71A4E89BEB68C4B476C5" }, { "b" : "7F9BAC76F000", "path" : "/lib64/libssh2.so.1", "elfType" : 3, "buildId" : "689404B6B895EACFAD65BA16E07E5BF0004F5E0C" }, { "b" : "7F9
      BAC51D000", "path" : "/lib64/libssl3.so", "elfType" : 3, "buildId" : "2E28F6A705F2ECEA8460D4716D5D1C24B5DDA5E4" }, { "b" : "7F9BAC2F6000", "path" : "/lib64/libsmime3.so", "elfType" : 3, "buildId" : "8D0B4010959
      C321022DF9CE239277A9D7B34A76A" }, { "b" : "7F9BABFC9000", "path" : "/lib64/libnss3.so", "elfType" : 3, "buildId" : "F5A64BB37FA3972E545EF459A51310F0AB56FA56" }, { "b" : "7F9BABD99000", "path" : "/lib64/libnssut
      il3.so", "elfType" : 3, "buildId" : "E0705772325A52C3372FFFB8BDE5F786E2E200D6" }, { "b" : "7F9BABB95000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "084D2194302908913F68B9DCD27DE46FA5B50522" }, 
      { "b" : "7F9BAB990000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "799B28AD9A5460D78376E2C11260F2E858B95DE3" }, { "b" : "7F9BAB752000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "
      DE762A28174110911B273E175D54F222B313CFE0" }, { "b" : "7F9BAB505000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "BCC30853830CD911E58700591830DF51ABCBD7BA" }, { "b" : "7F9BAB21C000", "path
      " : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "45BAB0BB455BDFA960FDA22E4124CF17B67CC930" }, { "b" : "7F9BAAFE9000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "A9B3906192687CC45D483A
      E3C58C8AF745A6726A" }, { "b" : "7F9BAADE5000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "B4BE1023D9606A88169DF411BF94AF417D7BA1A0" }, { "b" : "7F9BAABD6000", "path" : "/lib64/liblber-2.4.so
      .2", "elfType" : 3, "buildId" : "3192C56CD451E18EB9F29CB045432BA9C738DD29" }, { "b" : "7F9BAA981000", "path" : "/lib64/libldap-2.4.so.2", "elfType" : 3, "buildId" : "F1FADDDE0D21D5F4E2DCADEDD3B85B6E7AAC9883" },
       { "b" : "7F9BAA76B000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "B9D5F73428BD6AD68C96986B57BEA3B7CEDB9745" }, { "b" : "7F9BAA55B000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buil
      dId" : "94B3BCB669126166B77CDCE6092679A6AA2004C8" }, { "b" : "7F9BAA357000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F9BAA13A000", 
      "path" : "/lib64/libsasl2.so.3", "elfType" : 3, "buildId" : "E2F2017F821DD1B9D307DA1A9B8014F2941AEB7B" }, { "b" : "7F9BA9F13000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "D2DD4DA3FDE1477D2
      5BFFF80F3A25FDB541A8179" }, { "b" : "7F9BA9CDC000", "path" : "/lib64/libcrypt.so.1", "elfType" : 3, "buildId" : "740CAD898E29E1F3B73A323CCEC4A7C88911647F" }, { "b" : "7F9BA9A7A000", "path" : "/lib64/libpcre.so.
      1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }, { "b" : "7F9BA9877000", "path" : "/lib64/libfreebl3.so", "elfType" : 3, "buildId" : "B758881F4B6AF6C28C07A1A57713CBD2144628D4" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55a7dc251fe1]
       mongod(+0x23C21F9) [0x55a7dc2511f9]
       mongod(+0x23C26DD) [0x55a7dc2516dd]
       libpthread.so.0(+0xF5D0) [0x7f9bacfa85d0]
       libc.so.6(gsignal+0x37) [0x7f9bacc02207]
       libc.so.6(abort+0x148) [0x7f9bacc038f8]
       mongod(_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j+0x0) [0x55a7da89391a]
       mongod(_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x26A5) [0x55a7db0d2875]
       mongod(_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x1187) [0x55a7db0d1357]
       mongod(_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x67F) [0x55a7db0d084f]
       mongod(_ZN5mongo11buildStagesEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x144F) [0x55a7db0d161f]
       mongod(_ZN5mongo12StageBuilder5buildEPNS_16OperationContextEPNS_10CollectionERKNS_14CanonicalQueryERKNS_13QuerySolutionEPNS_10WorkingSetEPPNS_9PlanStageE+0x37) [0x55a7db0d31c7]
       mongod(_ZN5mongo15CachedPlanStage6replanEPNS_15PlanYieldPolicyEb+0x322) [0x55a7db03ebf2]
       mongod(_ZN5mongo15CachedPlanStage12pickBestPlanEPNS_15PlanYieldPolicyE+0x6D7) [0x55a7db040387]
       mongod(_ZN5mongo12PlanExecutor12pickBestPlanEPKNS_10CollectionE+0xD2) [0x55a7db0c5cc2]
       mongod(_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextESt10unique_ptrINS_10WorkingSetESt14default_deleteIS4_EES3_INS_9PlanStageES5_IS8_EES3_INS_13QuerySolutionES5_ISB_EES3_INS_14CanonicalQueryES5_ISE_EEPKN
      S_10CollectionENS_15NamespaceStringENS0_11YieldPolicyE+0x404) [0x55a7db0c9544]
       mongod(_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextESt10unique_ptrINS_10WorkingSetESt14default_deleteIS4_EES3_INS_9PlanStageES5_IS8_EES3_INS_13QuerySolutionES5_ISB_EES3_INS_14CanonicalQueryES5_ISE_EEPKN
      S_10CollectionENS0_11YieldPolicyE+0xA4) [0x55a7db0ca5f4]
       mongod(_ZN5mongo11getExecutorEPNS_16OperationContextEPNS_10CollectionESt10unique_ptrINS_14CanonicalQueryESt14default_deleteIS5_EENS_12PlanExecutor11YieldPolicyEm+0x117) [0x55a7db0bf117]
       mongod(+0x12306C6) [0x55a7db0bf6c6]
       mongod(_ZN5mongo15getExecutorFindEPNS_16OperationContextEPNS_10CollectionERKNS_15NamespaceStringESt10unique_ptrINS_14CanonicalQueryESt14default_deleteIS8_EEm+0x105) [0x55a7db0bff15]
       mongod(+0xFFE42A) [0x55a7dae8d42a]
       mongod(_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_19CommandReplyBuilderE+0xD9) [0x55a7dbcc0e99]
       mongod(+0xAAF0CF) [0x55a7da93e0cf]
       mongod(+0xAB1312) [0x55a7da940312]
       mongod(+0xAB3689) [0x55a7da942689]
       mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x3D1) [0x55a7da9435d1]
       mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x55a7da930bba]
       mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xBA) [0x55a7da93b95a]
       mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x55a7da936617]
       mongod(+0xAAAE21) [0x55a7da939e21]
       mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x55a7dbaf3432]
       mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x15F) [0x55a7da93482f]
       mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xAF5) [0x55a7da9379c5]
       mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x357) [0x55a7da935d57]
       mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x55a7da93669d]
       mongod(+0xAAAE21) [0x55a7da939e21]
       mongod(+0x1C64995) [0x55a7dbaf3995]
       mongod(+0x231BFB4) [0x55a7dc1aafb4]
       libpthread.so.0(+0x7DD5) [0x7f9bacfa0dd5]
       libc.so.6(clone+0x6D) [0x7f9baccc9ead]
      ----- END BACKTRACE -----
      

            Assignee:
            daniel.hatcher@mongodb.com Danny Hatcher (Inactive)
            Reporter:
            648813099@qq.com aa aaaaa
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: