[SERVER-60685] TransactionCoordinator may interrupt locally executing update with non-Interruption error category, leading to server crash Created: 14/Oct/21  Updated: 29/Oct/23  Resolved: 22/Nov/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.2.0, 4.4.0, 5.0.0, 5.1.0-rc0
Fix Version/s: 5.2.0, 5.1.2, 5.0.6, 4.4.11, 4.2.19

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Luis Osta (Inactive)
Resolution: Fixed Votes: 0
Labels: sharding-nyc-subteam1, sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-55379 Invariant failure _requests.empty() a... Closed
Problem/Incident
causes SERVER-61816 cancel_coordinate_txn_commit_with_tic... Closed
Related
is related to SERVER-56251 Alleviate problems that arise when Op... Backlog
is related to SERVER-60682 TransactionCoordinator may block acqu... Closed
is related to SERVER-37881 Coordinator should time out waiting f... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.1, v5.0, v4.4, v4.2
Sprint: Sharding 2021-11-15, Sharding 2021-11-29
Participants:
Case:
Linked BF Score: 170
Story Points: 2

 Description   

TransactionCoordinator schedules a task which will interrupt the OperationContext with a TransactionCoordinatorReachedAbortDecision error code if the coordinator has yet to reach its decision within transactionLifetimeLimitSeconds. The TransactionCoordinatorReachedAbortDecision error code isn't part of the Interruption error category. This means if the OperationContext for the update to write the participant list is interrupted, ReplClientInfo::setLastOpToSystemLastOpTimeIgnoringInterrupt() propagates the exception and leaves it uncaught in the LastOpFixer::~LastOpFixer() destructor.

Thread 74 "Transac.dinator" received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0x7f1c76785700 (LWP 32167)]
0x00007f1ca5538817 in raise () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) bt
#0  0x00007f1ca5538817 in raise () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007f1ca749ea0b in mongo::breakpoint () at src/mongo/util/debugger.cpp:72
#2  0x00007f1ca7472b33 in mongo::(anonymous namespace)::myTerminate () at src/mongo/util/signal_handlers_synchronous.cpp:226
#3  0x00007f1ca7566d66 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../src/combined/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x00007f1ca7568639 in __cxa_call_terminate (ue_header=ue_header@entry=0x55d314320000) at ../../../../src/combined/libstdc++-v3/libsupc++/eh_call.cc:54
#5  0x00007f1ca75669f5 in __cxxabiv1::__gxx_personality_v0 (version=<optimized out>, actions=6, exception_class=5138137972254386944, ue_header=0x55d314320000, context=<optimized out>) at ../../../../src/combined/libstdc++-v3/libsupc++/eh_personality.cc:676
#6  0x00007f1ca5755573 in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#7  0x00007f1ca5755df5 in _Unwind_Resume () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#8  0x00007f1ca115ec03 in boost::intrusive_ptr<mongo::Status::ErrorInfo const>::~intrusive_ptr (this=<optimized out>, __in_chrg=<optimized out>) at src/third_party/boost/boost/smart_ptr/intrusive_ptr.hpp:96
#9  mongo::Status::~Status (this=<optimized out>, __in_chrg=<optimized out>) at src/mongo/base/status.h:60
#10 mongo::StatusWith<mongo::repl::OpTime>::~StatusWith (this=<optimized out>, __in_chrg=<optimized out>) at src/mongo/base/status_with.h:81
#11 mongo::repl::ReplClientInfo::setLastOpToSystemLastOpTime (this=0x55d31400dc98, opCtx=0x55d314315200) at src/mongo/db/repl/repl_client_info.cpp:72
#12 0x00007f1ca1160b4f in mongo::repl::ReplClientInfo::setLastOpToSystemLastOpTimeIgnoringInterrupt (this=<optimized out>, opCtx=<optimized out>) at /opt/mongodbtoolchain/revisions/ba5f698948588cb5da922d3cadee990f5f9f48cd/stow/gcc-v3.pPo/include/c++/8.5.0/bits/unique_ptr.h:597
#13 0x00007f1ca304aca0 in mongo::write_ops_exec::(anonymous namespace)::LastOpFixer::~LastOpFixer (this=0x7f1c7677f2e0, __in_chrg=<optimized out>) at /opt/mongodbtoolchain/revisions/ba5f698948588cb5da922d3cadee990f5f9f48cd/stow/gcc-v3.pPo/include/c++/8.5.0/bits/unique_ptr.h:597
#14 0x00007f1ca30515fb in mongo::write_ops_exec::performUpdates (opCtx=opCtx@entry=0x55d314315200, wholeOp=..., source=<optimized out>) at src/mongo/db/ops/write_ops_exec.cpp:978
#15 0x00007f1c9d85cb99 in mongo::(anonymous namespace)::CmdUpdate::Invocation::typedRun (this=0x55d31448ab00, opCtx=0x55d314315200) at src/mongo/db/commands/write_commands.cpp:1412
#16 0x00007f1c9d85d7be in mongo::TypedCommand<mongo::(anonymous namespace)::CmdUpdate>::InvocationBase::_callTypedRun (opCtx=<optimized out>, this=<optimized out>) at src/mongo/db/commands.h:1256
#17 mongo::TypedCommand<mongo::(anonymous namespace)::CmdUpdate>::InvocationBase::_runImpl (reply=0x55d3142f87c0, opCtx=<optimized out>, this=<optimized out>) at src/mongo/db/commands.h:1257
#18 mongo::TypedCommand<mongo::(anonymous namespace)::CmdUpdate>::InvocationBase::run (this=<optimized out>, opCtx=<optimized out>, reply=0x55d3142f87c0) at src/mongo/db/commands.h:1262
#19 0x00007f1ca07eb22f in mongo::CommandHelpers::runCommandInvocation (opCtx=0x55d314315200, request=..., invocation=0x55d31448ab00, response=0x55d3142f87c0) at src/mongo/db/commands.cpp:199
#20 0x00007f1ca07ef09e in mongo::CommandHelpers::<lambda()>::operator() (__closure=0x7f1c76780d50) at src/mongo/db/commands.cpp:183
#21 mongo::makeReadyFutureWith<mongo::CommandHelpers::runCommandInvocation(std::shared_ptr<mongo::RequestExecutionContext>, std::shared_ptr<mongo::CommandInvocation>, mongo::transport::ServiceExecutor::ThreadingModel)::<lambda()> > (func=...) at src/mongo/util/future.h:1222
#22 mongo::CommandHelpers::runCommandInvocation (rec=std::shared_ptr<mongo::RequestExecutionContext> (use count 11, weak count 0) = {...}, invocation=std::shared_ptr<mongo::CommandInvocation> (use count 3, weak count 0) = {...}, threadingModel=<optimized out>) at src/mongo/db/commands.cpp:184
...
#88 0x00007f1ca201d2b4 in mongo::DBDirectClient::call (this=<optimized out>, toSend=..., response=..., assertOk=<optimized out>, actualServer=<optimized out>) at src/mongo/db/dbdirectclient.cpp:141
#89 0x00007f1ca04d2668 in mongo::DBClientBase::runCommandWithTarget (this=0x7f1c767832c0, request=...) at src/mongo/client/dbclient_base.cpp:240
#90 0x00007f1ca47f5fa3 in mongo::DBClientBase::runCommand (this=<optimized out>, request=...) at /opt/mongodbtoolchain/revisions/ba5f698948588cb5da922d3cadee990f5f9f48cd/stow/gcc-v3.pPo/include/c++/8.5.0/bits/move.h:182
#91 0x00007f1ca435984e in mongo::txn::(anonymous namespace)::persistParticipantListBlocking (opCtx=0x55d314315200, lsid=..., txnNumberAndRetryCounter=..., participantList=std::vector of length 2, capacity 2 = {...}) at src/third_party/boost/boost/smart_ptr/intrusive_ptr.hpp:96



 Comments   
Comment by Githook User [ 10/Jan/22 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-61816 Add steps past kWaitingForVotes to assert.soon

(cherry picked from commit 6d1b572c7ddbba652ffa49dc3783fbd27cec9714)

SERVER-60685 Add Interruption category to 'TransactionCoordinatorReachedAbortDecision'

(cherry picked from commit 78ab98a46b53582a5e69424bbb92f25c483fec0a)
Branch: v4.2
https://github.com/mongodb/mongo/commit/f2fc34bdec288cf3b90bf926d6a6c77631f4fa10

Comment by Githook User [ 10/Jan/22 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-60685 Add v4.2 compatibility to test
Branch: v4.4
https://github.com/mongodb/mongo/commit/e46858249a8449e9589bbb926ad367d092d71acc

Comment by Githook User [ 17/Dec/21 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-60685 Allow currentOp to work with both v4.4 and v5.0
Branch: v5.0
https://github.com/mongodb/mongo/commit/3ce368e255059dedf26874691d1a7797af5752be

Comment by Githook User [ 13/Dec/21 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-61816 Add steps past kWaitingForVotes to assert.soon

(cherry picked from commit 6d1b572c7ddbba652ffa49dc3783fbd27cec9714)

SERVER-60685 Add Interruption category to 'TransactionCoordinatorReachedAbortDecision'

(cherry picked from commit 78ab98a46b53582a5e69424bbb92f25c483fec0a)
Branch: v4.4
https://github.com/mongodb/mongo/commit/172eefd07a93db431d762e1ab017b21db9d10f9f

Comment by Githook User [ 08/Dec/21 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-61816 Add steps past kWaitingForVotes to assert.soon

(cherry picked from commit 6d1b572c7ddbba652ffa49dc3783fbd27cec9714)

SERVER-60685 Add Interruption category to 'TransactionCoordinatorReachedAbortDecision'

(cherry picked from commit 78ab98a46b53582a5e69424bbb92f25c483fec0a)
(cherry picked from commit 7634ffa5d056aa5efcc12079d00da898e6f258fb)
Branch: v5.0
https://github.com/mongodb/mongo/commit/f33c1dac76e2799d50b3453eaf14d771dc9646ab

Comment by Githook User [ 07/Dec/21 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-61816 Add steps past kWaitingForVotes to assert.soon

(cherry picked from commit 6d1b572c7ddbba652ffa49dc3783fbd27cec9714)

SERVER-60685 Add Interruption category to 'TransactionCoordinatorReachedAbortDecision'

(cherry picked from commit 78ab98a46b53582a5e69424bbb92f25c483fec0a)
Branch: v5.1
https://github.com/mongodb/mongo/commit/7634ffa5d056aa5efcc12079d00da898e6f258fb

Comment by Githook User [ 22/Nov/21 ]

Author:

{'name': 'Luis Osta', 'email': 'luis.osta@mongodb.com', 'username': 'LuisOsta'}

Message: SERVER-60685 Add Interruption category to 'TransactionCoordinatorReachedAbortDecision'
Branch: master
https://github.com/mongodb/mongo/commit/78ab98a46b53582a5e69424bbb92f25c483fec0a

Comment by Dmitry Agranat [ 14/Oct/21 ]

backtrace:

{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31431,   "ctx":"TransactionCoordinator","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"55D0328B0E6A","b":"55D02F9AA000","o":"2F06E6A","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"},{"a":"55D0328B28F9","b":"55D02F9AA000","o":"2F088F9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"55D0328AFAC6","b":"55D02F9AA000","o":"2F05AC6","s":"_ZN5mongo12_GLOBAL__N_111myTerminateEv","s+":"A6"},{"a":"55D032A3EF36","b":"55D02F9AA000","o":"3094F36","s":"_ZN10__cxxabiv111__terminateEPFvvE","s+":"6"},{"a":"55D032AD2F69","b":"55D02F9AA000","o":"3128F69","s":"__cxa_call_terminate","s+":"39"},{"a":"55D032A3E925","b":"55D02F9AA000","o":"3094925","s":"__gxx_personality_v0","s+":"275"},{"a":"7FD1905108E3","b":"7FD190501000","o":"F8E3","s":"_Unwind_GetTextRelBase","s+":"23E3"},{"a":"7FD190510E17","b":"7FD190501000","o":"FE17","s":"_Unwind_Resume","s+":"57"},{"a":"55D0308DCAFF","b":"55D02F9AA000","o":"F32AFF","s":"_ZN5mongo4repl14ReplClientInfo27setLastOpToSystemLastOpTimeEPNS_16OperationContextE.cold.366","s+":"45"},{"a":"55D032569A5C","b":"55D02F9AA000","o":"2BBFA5C","s":"_ZN5mongo4repl14ReplClientInfo44setLastOpToSystemLastOpTimeIgnoringInterruptEPNS_16OperationContextE","s+":"2C"},{"a":"55D0312B7A7D","b":"55D02F9AA000","o":"190DA7D","s":"_ZN5mongo12_GLOBAL__N_111LastOpFixerD2Ev","s+":"6D"},{"a":"55D0312B8CB1","b":"55D02F9AA000","o":"190ECB1","s":"_ZN5mongo14performUpdatesEPNS_16OperationContextERKNS_9write_ops6UpdateE","s+":"1181"},{"a":"55D031212A16","b":"55D02F9AA000","o":"1868A16","s":"_ZNK5mongo12_GLOBAL__N_19CmdUpdate10Invocation7runImplEPNS_16OperationContextERNS_14BSONObjBuilderE","s+":"46"},{"a":"55D03121058D","b":"55D02F9AA000","o":"186658D","s":"_ZN5mongo12_GLOBAL__N_112WriteCommand14InvocationBase3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE","s+":"15D"},{"a":"55D0317D79BF","b":"55D02F9AA000","o":"1E2D9BF","s":"_ZN5mongo14CommandHelpers20runCommandInvocationEPNS_16OperationContextERKNS_12OpMsgRequestEPNS_17CommandInvocationEPNS_3rpc21ReplyBuilderInterfaceE","s+":"7F"},{"a":"55D030F0EF5F","b":"55D02F9AA000","o":"1564F5F","s":"_ZN5mongo12_GLOBAL__N_114runCommandImplEPNS_16OperationContextEPNS_17CommandInvocationERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceENS_11LogicalTimeERKNS_23ServiceEntryPointCommon5HooksEPNS_14BSONObjBuilderERKNS_30OperationSessionInfoFromClientE","s+":"9FF"},{"a":"55D030F1219F","b":"55D02F9AA000","o":"156819F","s":"_ZZN5mongo12_GLOBAL__N_116receivedCommandsEPNS_16OperationContextERKNS_7MessageERKNS_23ServiceEntryPointCommon5HooksEENKUlvE_clEv","s+":"1E8F"},{"a":"55D030F1304B","b":"55D02F9AA000","o":"156904B","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE","s+":"52B"},{"a":"55D030F00F8C","b":"55D02F9AA000","o":"1556F8C","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE","s+":"3C"},{"a":"55D0315DB749","b":"55D02F9AA000","o":"1C31749","s":"_ZN5mongo12_GLOBAL__N_121loopbackBuildResponseEPNS_16OperationContextEPNS_9LastErrorERNS_7MessageE","s+":"359"},{"a":"55D0315DBCA8","b":"55D02F9AA000","o":"1C31CA8","s":"_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"38"},{"a":"55D03236B5C2","b":"55D02F9AA000","o":"29C15C2","s":"_ZN5mongo12DBClientBase20runCommandWithTargetENS_12OpMsgRequestE","s+":"1F2"},{"a":"55D03103FF2F","b":"55D02F9AA000","o":"1695F2F","s":"_ZN5mongo12DBClientBase10runCommandENS_12OpMsgRequestE","s+":"7F"},{"a":"55D031113E11","b":"55D02F9AA000","o":"1769E11","s":"_ZN5mongo3txn12_GLOBAL__N_130persistParticipantListBlockingEPNS_16OperationContextERKNS_16LogicalSessionIdElRKSt6vectorINS_7ShardIdESaIS8_EE","s+":"A61"},{"a":"55D03111755A","b":"55D02F9AA000","o":"176D55A","s":"_ZZN5mongo15unique_functionIFvRKNS_8executor12TaskExecutor12CallbackArgsEEE8makeImplIZNS_3txn18AsyncWorkScheduler14scheduleWorkAtIZZNS9_23persistParticipantsListERSA_RKNS_16LogicalSessionIdElRKSt6vectorINS_7ShardIdESaISH_EEENKUlvE0_clEvEUlPNS_16OperationContextEE_EENS_6FutureINS_14future_details17UnwrappedTypeImplINSt13invoke_resultIT_JOSO_EE4typeEE4typeEEENS_6Date_tEOSU_EUlS5_E_EEDaS12_EN12SpecificImpl4callES5_","s+":"15A"},{"a":"55D03222E383","b":"55D02F9AA000","o":"2884383","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor11runCallbackESt10shared_ptrINS1_13CallbackStateEE","s+":"113"},{"a":"55D03222E792","b":"55D02F9AA000","o":"2884792","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS6_13CallbackStateEESaISB_EEERKSt14_List_iteratorISB_ESI_St11unique_lockINS_12latch_detail5LatchEEEUlT_E1_EEDaOSN_EN12SpecificImpl4callEOS1_","s+":"A2"},{"a":"55D0323DC7D2","b":"55D02F9AA000","o":"2A327D2","s":"_ZN5mongo10ThreadPool10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"132"},{"a":"55D0323DEE16","b":"55D02F9AA000","o":"2A34E16","s":"_ZN5mongo10ThreadPool13_consumeTasksEv","s+":"86"},{"a":"55D0323DFBC1","b":"55D02F9AA000","o":"2A35BC1","s":"_ZN5mongo10ThreadPool17_workerThreadBodyEPS0_RKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"E1"},{"a":"55D0323DFEF0","b":"55D02F9AA000","o":"2A35EF0","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"},{"a":"55D032A5AA4F","b":"55D02F9AA000","o":"30B0A4F","s":"execute_native_thread_routine","s+":"F"},{"a":"7FD1902ECEA5","b":"7FD1902E5000","o":"7EA5","s":"start_thread","s+":"C5"},{"a":"7FD1900159FD","b":"7FD18FF17000","o":"FE9FD","s":"clone","s+":"6D"}],"processInfo":{"mongodbVersion":"4.4.9","gitVersion":"b4048e19814bfebac717cf5a880076aa69aba481","compiledModules":["enterprise"],"uname":{"sysname":"Linux","release":"3.10.0-1160.41.1.el7.x86_64","version":"#1 SMP Tue Aug 31 14:52:47 UTC 2021","machine":"x86_64"},"somap":[{"b":"55D02F9AA000","elfType":3,"buildId":"0262133647F37291A8A61E6B6358AF48E14386DA"},{"b":"7FD190501000","path":"/lib64/libgcc_s.so.1","elfType":3,"buildId":"EDF51350C7F71496149D064AA8B1441F786DF88A"},{"b":"7FD1902E5000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"E10CC8F2B932FC3DAEDA22F8DAC5EBB969524E5B"},{"b":"7FD18FF17000","path":"/lib64/libc.so.6","elfType":3,"buildId":"A317B42B15368ADCAE21C11107691A03EC91059D"}]}}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0328B0E6A","b":"55D02F9AA000","o":"2F06E6A","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0328B28F9","b":"55D02F9AA000","o":"2F088F9","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0328AFAC6","b":"55D02F9AA000","o":"2F05AC6","s":"_ZN5mongo12_GLOBAL__N_111myTerminateEv","s+":"A6"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D032A3EF36","b":"55D02F9AA000","o":"3094F36","s":"_ZN10__cxxabiv111__terminateEPFvvE","s+":"6"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D032AD2F69","b":"55D02F9AA000","o":"3128F69","s":"__cxa_call_terminate","s+":"39"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D032A3E925","b":"55D02F9AA000","o":"3094925","s":"__gxx_personality_v0","s+":"275"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"7FD1905108E3","b":"7FD190501000","o":"F8E3","s":"_Unwind_GetTextRelBase","s+":"23E3"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"7FD190510E17","b":"7FD190501000","o":"FE17","s":"_Unwind_Resume","s+":"57"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0308DCAFF","b":"55D02F9AA000","o":"F32AFF","s":"_ZN5mongo4repl14ReplClientInfo27setLastOpToSystemLastOpTimeEPNS_16OperationContextE.cold.366","s+":"45"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D032569A5C","b":"55D02F9AA000","o":"2BBFA5C","s":"_ZN5mongo4repl14ReplClientInfo44setLastOpToSystemLastOpTimeIgnoringInterruptEPNS_16OperationContextE","s+":"2C"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0312B7A7D","b":"55D02F9AA000","o":"190DA7D","s":"_ZN5mongo12_GLOBAL__N_111LastOpFixerD2Ev","s+":"6D"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0312B8CB1","b":"55D02F9AA000","o":"190ECB1","s":"_ZN5mongo14performUpdatesEPNS_16OperationContextERKNS_9write_ops6UpdateE","s+":"1181"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D031212A16","b":"55D02F9AA000","o":"1868A16","s":"_ZNK5mongo12_GLOBAL__N_19CmdUpdate10Invocation7runImplEPNS_16OperationContextERNS_14BSONObjBuilderE","s+":"46"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03121058D","b":"55D02F9AA000","o":"186658D","s":"_ZN5mongo12_GLOBAL__N_112WriteCommand14InvocationBase3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE","s+":"15D"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0317D79BF","b":"55D02F9AA000","o":"1E2D9BF","s":"_ZN5mongo14CommandHelpers20runCommandInvocationEPNS_16OperationContextERKNS_12OpMsgRequestEPNS_17CommandInvocationEPNS_3rpc21ReplyBuilderInterfaceE","s+":"7F"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D030F0EF5F","b":"55D02F9AA000","o":"1564F5F","s":"_ZN5mongo12_GLOBAL__N_114runCommandImplEPNS_16OperationContextEPNS_17CommandInvocationERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceENS_11LogicalTimeERKNS_23ServiceEntryPointCommon5HooksEPNS_14BSONObjBuilderERKNS_30OperationSessionInfoFromClientE","s+":"9FF"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D030F1219F","b":"55D02F9AA000","o":"156819F","s":"_ZZN5mongo12_GLOBAL__N_116receivedCommandsEPNS_16OperationContextERKNS_7MessageERKNS_23ServiceEntryPointCommon5HooksEENKUlvE_clEv","s+":"1E8F"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D030F1304B","b":"55D02F9AA000","o":"156904B","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE","s+":"52B"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D030F00F8C","b":"55D02F9AA000","o":"1556F8C","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE","s+":"3C"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0315DB749","b":"55D02F9AA000","o":"1C31749","s":"_ZN5mongo12_GLOBAL__N_121loopbackBuildResponseEPNS_16OperationContextEPNS_9LastErrorERNS_7MessageE","s+":"359"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0315DBCA8","b":"55D02F9AA000","o":"1C31CA8","s":"_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"38"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03236B5C2","b":"55D02F9AA000","o":"29C15C2","s":"_ZN5mongo12DBClientBase20runCommandWithTargetENS_12OpMsgRequestE","s+":"1F2"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03103FF2F","b":"55D02F9AA000","o":"1695F2F","s":"_ZN5mongo12DBClientBase10runCommandENS_12OpMsgRequestE","s+":"7F"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D031113E11","b":"55D02F9AA000","o":"1769E11","s":"_ZN5mongo3txn12_GLOBAL__N_130persistParticipantListBlockingEPNS_16OperationContextERKNS_16LogicalSessionIdElRKSt6vectorINS_7ShardIdESaIS8_EE","s+":"A61"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03111755A","b":"55D02F9AA000","o":"176D55A","s":"_ZZN5mongo15unique_functionIFvRKNS_8executor12TaskExecutor12CallbackArgsEEE8makeImplIZNS_3txn18AsyncWorkScheduler14scheduleWorkAtIZZNS9_23persistParticipantsListERSA_RKNS_16LogicalSessionIdElRKSt6vectorINS_7ShardIdESaISH_EEENKUlvE0_clEvEUlPNS_16OperationContextEE_EENS_6FutureINS_14future_details17UnwrappedTypeImplINSt13invoke_resultIT_JOSO_EE4typeEE4typeEEENS_6Date_tEOSU_EUlS5_E_EEDaS12_EN12SpecificImpl4callES5_","s+":"15A"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03222E383","b":"55D02F9AA000","o":"2884383","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor11runCallbackESt10shared_ptrINS1_13CallbackStateEE","s+":"113"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D03222E792","b":"55D02F9AA000","o":"2884792","s":"_ZZN5mongo15unique_functionIFvNS_6StatusEEE8makeImplIZNS_8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS6_13CallbackStateEESaISB_EEERKSt14_List_iteratorISB_ESI_St11unique_lockINS_12latch_detail5LatchEEEUlT_E1_EEDaOSN_EN12SpecificImpl4callEOS1_","s+":"A2"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0323DC7D2","b":"55D02F9AA000","o":"2A327D2","s":"_ZN5mongo10ThreadPool10_doOneTaskEPSt11unique_lockINS_12latch_detail5LatchEE","s+":"132"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0323DEE16","b":"55D02F9AA000","o":"2A34E16","s":"_ZN5mongo10ThreadPool13_consumeTasksEv","s+":"86"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0323DFBC1","b":"55D02F9AA000","o":"2A35BC1","s":"_ZN5mongo10ThreadPool17_workerThreadBodyEPS0_RKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"E1"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D0323DFEF0","b":"55D02F9AA000","o":"2A35EF0","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_10ThreadPool25_startWorkerThread_inlockEvEUlvE2_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"60"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D032A5AA4F","b":"55D02F9AA000","o":"30B0A4F","s":"execute_native_thread_routine","s+":"F"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"7FD1902ECEA5","b":"7FD1902E5000","o":"7EA5","s":"start_thread","s+":"C5"}}}
{"t":{"$date":"2021-10-06T20:01:08.455+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"TransactionCoordinator","msg":"  Frame: {frame}","attr":{"frame":{"a":"7FD1900159FD","b":"7FD18FF17000","o":"FE9FD","s":"clone","s+":"6D"}}}

Generated at Thu Feb 08 05:50:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.