[SERVER-57610] IndexAccessMethod::commitBulk() retries inserting index keys on WriteConflictException Created: 10/Jun/21  Updated: 29/Oct/23  Resolved: 11/Jun/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.0-rc2, 5.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
is related to SERVER-59091 Avoid fetching namespace from durable... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: Execution Team 2021-06-14, Execution Team 2021-06-28
Participants:
Linked BF Score: 40

 Description   

This is need to fix a build failure occurring on ephemeralForTest

[j0] | 2021-05-31T22:26:32.572+00:00 I  STORAGE  20649   [IndexBuildsCoordinatorMongod-5] "Index build: failed","attr":{"buildUUID":{"uuid":{"$uuid":"f8444ea1-92ff-4ddc-96fe-3b68f853c199"}},"collectionUUID":{"uuid":{"$uuid":"f50a5e4a-001d-458d-a07a-a7492e8497a9"}},"namespace":"test.wildcard_index_bounds","error":{"code":112,"codeName":"WriteConflict","errmsg":"WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction."}}
[j0] | 2021-05-31T22:26:32.572+00:00 I  COMMAND  51803   [conn1507] "Slow query","attr":{"type":"command","ns":"test.$cmd","appName":"MongoDB Shell","command":{"delete":"remove6","ordered":true,"lsid":{"id":{"$uuid":"214475e1-fcce-49c7-a095-a948702c9ad2"}},"$db":"test"},"numYields":14,"reslen":45,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":15}},"ReplicationStateTransition":{"acquireCount":{"w":16}},"Global":{"acquireCount":{"r":1,"w":15}},"Database":{"acquireCount":{"w":15}},"Collection":{"acquireCount":{"w":15}},"Mutex":{"acquireCount":{"r":1}}},"flowControl":{"acquireCount":15,"timeAcquiringMicros":58},"remote":"127.0.0.1:43436","protocol":"op_msg","durationMillis":158}
[j0] | 2021-05-31T22:26:32.572+00:00 F  -        23081   [IndexBuildsCoordinatorMongod-5] "Invariant failure","attr":{"expr":"status.isA<ErrorCategory::Interruption>() || status.isA<ErrorCategory::ShutdownError>()","msg":"Unexpected error code during index build cleanup: WriteConflict: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.","file":"src/mongo/db/index_builds_coordinator.cpp","line":2166}
[j0] | 2021-05-31T22:26:32.572+00:00 F  -        23082   [IndexBuildsCoordinatorMongod-5] "\n\n***aborting after invariant() failure\n\n"
[j0] | 2021-05-31T22:26:32.572+00:00 F  CONTROL  4757800 [IndexBuildsCoordinatorMongod-5] "Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}
[j0] | 2021-05-31T22:26:32.573+00:00 I  COMMAND  518070  [conn1507] "CMD: drop","attr":{"namespace":"test.remove6"}
[j0] | 2021-05-31T22:26:32.573+00:00 I  STORAGE  20318   [conn1507] "Finishing collection drop","attr":{"namespace":"test.remove6","uuid":{"uuid":{"$uuid":"03236555-dd32-496b-abf5-156970e0a96d"}}}
[j0] | 2021-05-31T22:26:32.601+00:00 I  STORAGE  20320   [conn1507] "createCollection","attr":{"namespace":"test.remove6","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"44b023a4-ceae-46ae-99ff-f00788db9f68"}},"options":{}}
[j0] | 2021-05-31T22:26:32.602+00:00 I  INDEX    20345   [conn1507] "Index build: done building","attr":{"buildUUID":null,"namespace":"test.remove6","index":"_id_","commitTimestamp":null}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31380   [IndexBuildsCoordinatorMongod-5] "BACKTRACE","attr":{"bt":{"backtrace":[{"a":"562E0F9D1E95","b":"562E0BC43000","o":"3D8EE95","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.357","s+":"215"},{"a":"562E0F9D48E9","b":"562E0BC43000","o":"3D918E9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"562E0F9CCFA6","b":"562E0BC43000","o":"3D89FA6","s":"abruptQuit","s+":"66"},{"a":"7F4AE9F05D80","b":"7F4AE9EF3000","o":"12D80","s":"funlockfile","s+":"50"},{"a":"7F4AE97E493F","b":"7F4AE97AD000","o":"3793F","s":"gsignal","s+":"10F"},{"a":"7F4AE97CEC95","b":"7F4AE97AD000","o":"21C95","s":"abort","s+":"127"},{"a":"562E0CFAE720","b":"562E0BC43000","o":"136B720","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j","s+":"113"},{"a":"562E0CCE7D05","b":"562E0BC43000","o":"10A4D05","s":"_ZN5mongo22IndexBuildsCoordinator19_runIndexBuildInnerEPNS_16OperationContextESt10shared_ptrINS_19ReplIndexBuildStateEERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE.cold.2362","s+":"CB"},{"a":"562E0DFDDBDB","b":"562E0BC43000","o":"239ABDB","s":"_ZN5mongo22IndexBuildsCoordinator14_runIndexBuildEPNS_16OperationContextERKNS_4UUIDERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE","s+":"1FB"},{"a":"562E0DD073C1","b":"562E0BC43000","o":"20C43C1","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_28IndexBuildsCoordinatorMongod16_startIndexBuildEPNS_16OperationContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4UUIDERKSt6vectorINS_7BSONObjESaISG_EERKSE_NS_18IndexBuildProtocolENS_22IndexBuildsCoordinator17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEEEUlT_E5_EEDaOSW_EN12SpecificImpl4callEOS1_","s+":"371"},{"a":"562E0F793275","b":"562E0BC43000","o":"3B50275","s":"_ZN5mongo10ThreadPool4Impl10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"135"},{"a":"562E0F79490B","b":"562E0BC43000","o":"3B5190B","s":"_ZN5mongo10ThreadPool4Impl13_consumeTasksEv","s+":"8B"},{"a":"562E0F795D11","b":"562E0BC43000","o":"3B52D11","s":"_ZN5mongo10ThreadPool4Impl17_workerThreadBodyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"161"},{"a":"562E0F796220","b":"562E0BC43000","o":"3B53220","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool4Impl25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"},{"a":"562E0FB745DF","b":"562E0BC43000","o":"3F315DF","s":"execute_native_thread_routine","s+":"F"},{"a":"7F4AE9EFB2DE","b":"7F4AE9EF3000","o":"82DE","s":"start_thread","s+":"FE"},{"a":"7F4AE98A9A63","b":"7F4AE97AD000","o":"FCA63","s":"clone","s+":"43"}],"processInfo":{"mongodbVersion":"5.0.0-alpha0-726-gfcc590c","gitVersion":"fcc590c63c9c08fa0a895765071c5462b19f3160","compiledModules":[],"uname":{"sysname":"Linux","release":"4.18.0-80.1.2.el8_0.x86_64","version":"#1 SMP Sun Apr 28 09:21:22 UTC 2019","machine":"x86_64"},"somap":[{"b":"562E0BC43000","elfType":3,"buildId":"02D67E172BC32C50FF59BE56ED869D88A0027DB2"},{"b":"7F4AE9EF3000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"5326B8728FA01B7149DAC943100F1405533E76CE"},{"b":"7F4AE97AD000","path":"/lib64/libc.so.6","elfType":3,"buildId":"0598B7D6A05E64AE676133CF6331AF5578888AD0"}]}}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9D1E95","b":"562E0BC43000","o":"3D8EE95","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.357","s+":"215"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9D48E9","b":"562E0BC43000","o":"3D918E9","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F9CCFA6","b":"562E0BC43000","o":"3D89FA6","s":"abruptQuit","s+":"66"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE9F05D80","b":"7F4AE9EF3000","o":"12D80","s":"funlockfile","s+":"50"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE97E493F","b":"7F4AE97AD000","o":"3793F","s":"gsignal","s+":"10F"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE97CEC95","b":"7F4AE97AD000","o":"21C95","s":"abort","s+":"127"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0CFAE720","b":"562E0BC43000","o":"136B720","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j","s+":"113"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0CCE7D05","b":"562E0BC43000","o":"10A4D05","s":"_ZN5mongo22IndexBuildsCoordinator19_runIndexBuildInnerEPNS_16OperationContextESt10shared_ptrINS_19ReplIndexBuildStateEERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE.cold.2362","s+":"CB"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0DFDDBDB","b":"562E0BC43000","o":"239ABDB","s":"_ZN5mongo22IndexBuildsCoordinator14_runIndexBuildEPNS_16OperationContextERKNS_4UUIDERKNS0_17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEE","s+":"1FB"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0DD073C1","b":"562E0BC43000","o":"20C43C1","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_28IndexBuildsCoordinatorMongod16_startIndexBuildEPNS_16OperationContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4UUIDERKSt6vectorINS_7BSONObjESaISG_EERKSE_NS_18IndexBuildProtocolENS_22IndexBuildsCoordinator17IndexBuildOptionsERKN5boost8optionalINS_15ResumeIndexInfoEEEEUlT_E5_EEDaOSW_EN12SpecificImpl4callEOS1_","s+":"371"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F793275","b":"562E0BC43000","o":"3B50275","s":"_ZN5mongo10ThreadPool4Impl10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"135"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F79490B","b":"562E0BC43000","o":"3B5190B","s":"_ZN5mongo10ThreadPool4Impl13_consumeTasksEv","s+":"8B"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F795D11","b":"562E0BC43000","o":"3B52D11","s":"_ZN5mongo10ThreadPool4Impl17_workerThreadBodyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"161"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0F796220","b":"562E0BC43000","o":"3B53220","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool4Impl25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"562E0FB745DF","b":"562E0BC43000","o":"3F315DF","s":"execute_native_thread_routine","s+":"F"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE9EFB2DE","b":"7F4AE9EF3000","o":"82DE","s":"start_thread","s+":"FE"}}
[j0] | 2021-05-31T22:26:32.635+00:00 I  CONTROL  31445   [IndexBuildsCoordinatorMongod-5] "Frame","attr":{"frame":{"a":"7F4AE98A9A63","b":"7F4AE97AD000","o":"FCA63","s":"clone","s+":"43"}}



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-57610 IndexAccessMethod::commitBulk() retries inserting index keys on WriteConflictException

(cherry picked from commit d984013330025d5b65dfdeabf8c72b0cba997204)
Branch: v5.0
https://github.com/mongodb/mongo/commit/37638ed089039a8b5b8954874b5f42844b5452b2

Comment by Githook User [ 11/Jun/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-57610 IndexAccessMethod::commitBulk() retries inserting index keys on WriteConflictException
Branch: master
https://github.com/mongodb/mongo/commit/d984013330025d5b65dfdeabf8c72b0cba997204

Generated at Thu Feb 08 05:42:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.