Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-57610

IndexAccessMethod::commitBulk() retries inserting index keys on WriteConflictException

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.0.0-rc2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • ALL
    • v5.0
    • Execution Team 2021-06-14, Execution Team 2021-06-28
    • 40

      This is need to fix a build failure occurring on ephemeralForTest

      [j0] | 2021-05-31T22:26:32.572+00:00 I  STORAGE  20649   [IndexBuildsCoordinatorMongod-5] "Index build: failed","attr":{"buildUUID":{"uuid":{"$uuid":"f8444ea1-92ff-4ddc-96fe-3b68f853c199"}},"collectionUUID":{"uuid":{"$uuid":"f50a5e4a-001d-458d-a07a-a7492e8497a9"}},"namespace":"test.wildcard_index_bounds","error":{"code":112,"codeName":"WriteConflict","errmsg":"WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction."}}
      [j0] | 2021-05-31T22:26:32.572+00:00 I  COMMAND  51803   [conn1507] "Slow query","attr":{"type":"command","ns":"test.$cmd","appName":"MongoDB Shell","command":{"delete":"remove6","ordered":true,"lsid":{"id":{"$uuid":"214475e1-fcce-49c7-a095-a948702c9ad2"}},"$db":"test"},"numYields":14,"reslen":45,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":15}},"ReplicationStateTransition":{"acquireCount":{"w":16}},"Global":{"acquireCount":{"r":1,"w":15}},"Database":{"acquireCount":{"w":15}},"Collection":{"acquireCount":{"w":15}},"Mutex":{"acquireCount":{"r":1}}},"flowControl":{"acquireCount":15,"timeAcquiringMicros":58},"remote":"127.0.0.1:43436","protocol":"op_msg","durationMillis":158}
      [j0] | 2021-05-31T22:26:32.572+00:00 F  -        23081   [IndexBuildsCoordinatorMongod-5] "Invariant failure","attr":{"expr":"status.isA<ErrorCategory::Interruption>() || status.isA<ErrorCategory::ShutdownError>()","msg":"Unexpected error code during index build cleanup: WriteConflict: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.","file":"src/mongo/db/index_builds_coordinator.cpp","line":2166}
      [j0] | 2021-05-31T22:26:32.572+00:00 F  -        23082   [IndexBuildsCoordinatorMongod-5] "\n\n***aborting after invariant() failure\n\n"
      [j0] | 2021-05-31T22:26:32.572+00:00 F  CONTROL  4757800 [IndexBuildsCoordinatorMongod-5] "Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}
      [j0] | 2021-05-31T22:26:32.573+00:00 I  COMMAND  518070  [conn1507] "CMD: drop","attr":{"namespace":"test.remove6"}
      [j0] | 2021-05-31T22:26:32.573+00:00 I  STORAGE  20318   [conn1507] "Finishing collection drop","attr":{"namespace":"test.remove6","uuid":{"uuid":{"$uuid":"03236555-dd32-496b-abf5-156970e0a96d"}}}
      [j0] | 2021-05-31T22:26:32.601+00:00 I  STORAGE  20320   [conn1507] "createCollection","attr":{"namespace":"test.remove6","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"44b023a4-ceae-46ae-99ff-f00788db9f68"}},"options":{}}
      [j0] | 2021-05-31T22:26:32.602+00:00 I  INDEX    20345   [conn1507] "Index build: done building","attr":{"buildUUID":null,"namespace":"test.remove6","index":"_id_","commitTimestamp":null}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31380   [IndexBuildsCoordinatorMongod-5] "BACKTRACE","attr":{"bt":{"backtrace":[{"a":"562E0F9D1E95","b":"562E0BC43000","o":"3D8EE95","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.357","s+":"215"},{"a":"562E0F9D48E9","b":"562E0BC43000","o":"3D918E9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"562E0F9CCFA6","b":"562E0BC43000","o":"3D89FA6","s":"abruptQuit","s+":"66"},{"a":"7F4AE9F05D80","b":"7F4AE9EF3000","o":"12D80","s":"funlockfile","s+":"50"},{"a":"7F4AE97E493F","b":"7F4AE97AD000","o":"3793F","s":"gsignal","s+":"10F"},{"a":"7F4AE97CEC95","b":"7F4AE97AD000","o":"21C95","s":"abort","s+":"127"},{"a":"562E0CFAE720","b":"562E0BC43000","o":"136B720","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j","s+":"113"},{"a":"562E0CCE7D05","b":"562E0BC43000","o":"10A4D05","s":"_ZN5mongo22IndexBuildsCoordinator19_runIndexBuildInnerEPNS_16OperationContextESt10shared_ptrINS_19ReplIndexBuildStateEERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE.cold.2362","s+":"CB"},{"a":"562E0DFDDBDB","b":"562E0BC43000","o":"239ABDB","s":"_ZN5mongo22IndexBuildsCoordinator14_runIndexBuildEPNS_16OperationContextERKNS_4UUIDERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE","s+":"1FB"},{"a":"562E0DD073C1","b":"562E0BC43000","o":"20C43C1","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_28IndexBuildsCoordinatorMongod16_startIndexBuildEPNS_16OperationContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4UUIDERKSt6vectorINS_7BSONObjESaISG_EERKSE_NS_18IndexBuildProtocolENS_22IndexBuildsCoordinator17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEEEUlT_E5_EEDaOSW_EN12SpecificImpl4callEOS1_","s+":"371"},{"a":"562E0F793275","b":"562E0BC43000","o":"3B50275","s":"_ZN5mongo10ThreadPool4Impl10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"135"},{"a":"562E0F79490B","b":"562E0BC43000","o":"3B5190B","s":"_ZN5mongo10ThreadPool4Impl13_consumeTasksEv","s+":"8B"},{"a":"562E0F795D11","b":"562E0BC43000","o":"3B52D11","s":"_ZN5mongo10ThreadPool4Impl17_workerThreadBodyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"161"},{"a":"562E0F796220","b":"562E0BC43000","o":"3B53220","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool4Impl25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"},{"a":"562E0FB745DF","b":"562E0BC43000","o":"3F315DF","s":"execute_native_thread_routine","s+":"F"},{"a":"7F4AE9EFB2DE","b":"7F4AE9EF3000","o":"82DE","s":"start_thread","s+":"FE"},{"a":"7F4AE98A9A63","b":"7F4AE97AD000","o":"FCA63","s":"clone","s+":"43"}],"processInfo":{"mongodbVersion":"5.0.0-alpha0-726-gfcc590c","gitVersion":"fcc590c63c9c08fa0a895765071c5462b19f3160","compiledModules":[],"uname":{"sysname":"Linux","release":"4.18.0-80.1.2.el8_0.x86_64","version":"#1 SMP Sun Apr 28 09:21:22 UTC 2019","machine":"x86_64"},"somap":[{"b":"562E0BC43000","elfType":3,"buildId":"02D67E172BC32C50FF59BE56ED869D88A0027DB2"},{"b":"7F4AE9EF3000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"5326B8728FA01B7149DAC943100F1405533E76CE"},{"b":"7F4AE97AD000","path":"/lib64/libc.so.6","elfType":3,"buildId":"0598B7D6A05E64AE676133CF6331AF5578888AD0"}]}}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9D1E95","b":"562E0BC43000","o":"3D8EE95","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.357","s+":"215"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9D48E9","b":"562E0BC43000","o":"3D918E9","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9CCFA6","b":"562E0BC43000","o":"3D89FA6","s":"abruptQuit","s+":"66"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE9F05D80","b":"7F4AE9EF3000","o":"12D80","s":"funlockfile","s+":"50"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE97E493F","b":"7F4AE97AD000","o":"3793F","s":"gsignal","s+":"10F"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE97CEC95","b":"7F4AE97AD000","o":"21C95","s":"abort","s+":"127"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0CFAE720","b":"562E0BC43000","o":"136B720","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j","s+":"113"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0CCE7D05","b":"562E0BC43000","o":"10A4D05","s":"_ZN5mongo22IndexBuildsCoordinator19_runIndexBuildInnerEPNS_16OperationContextESt10shared_ptrINS_19ReplIndexBuildStateEERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE.cold.2362","s+":"CB"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0DFDDBDB","b":"562E0BC43000","o":"239ABDB","s":"_ZN5mongo22IndexBuildsCoordinator14_runIndexBuildEPNS_16OperationContextERKNS_4UUIDERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE","s+":"1FB"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0DD073C1","b":"562E0BC43000","o":"20C43C1","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_28IndexBuildsCoordinatorMongod16_startIndexBuildEPNS_16OperationContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4UUIDERKSt6vectorINS_7BSONObjESaISG_EERKSE_NS_18IndexBuildProtocolENS_22IndexBuildsCoordinator17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEEEUlT_E5_EEDaOSW_EN12SpecificImpl4callEOS1_","s+":"371"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F793275","b":"562E0BC43000","o":"3B50275","s":"_ZN5mongo10ThreadPool4Impl10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"135"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F79490B","b":"562E0BC43000","o":"3B5190B","s":"_ZN5mongo10ThreadPool4Impl13_consumeTasksEv","s+":"8B"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F795D11","b":"562E0BC43000","o":"3B52D11","s":"_ZN5mongo10ThreadPool4Impl17_workerThreadBodyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"161"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F796220","b":"562E0BC43000","o":"3B53220","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool4Impl25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0FB745DF","b":"562E0BC43000","o":"3F315DF","s":"execute_native_thread_routine","s+":"F"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE9EFB2DE","b":"7F4AE9EF3000","o":"82DE","s":"start_thread","s+":"FE"}}
      [j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE98A9A63","b":"7F4AE97AD000","o":"FCA63","s":"clone","s+":"43"}}
      

            Assignee:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Reporter:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: