[SERVER-53950] tenant_migration_conflicting_recipient_sync_data_cmds failing with tenant_migration feature flag on for ASAN Builder Created: 21/Jan/21  Updated: 29/Oct/23  Resolved: 16/Feb/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.0

Type: Bug Priority: Major - P3
Reporter: Cristopher Stauffer Assignee: Andrew Shuvalov (Inactive)
Resolution: Fixed Votes: 0
Labels: pm-1791_other_required
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-54302 Write tenant migration test when the ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Sharding 2021-02-22
Participants:

 Description   

Task:

https://spruce.mongodb.com/task/mongodb_mongo_master_ubuntu1804_debug_asan_all_feature_flags_display_replica_sets_auth_patch_ab1ebf2353773101628de93855aa4cad3847c200_60087b74c9ec44515d0febb1_21_01_20_18_57_33/tests?execution=0

Log:

https://logkeeper.mongodb.org/lobster/build/5fe28ab1b698f8b0537e7e575dd62172/test/60088e4abe07c465fc57c561#l=1

Note: This is blocking enabling replset tasks for the ASAN builder.

Once resolved, uncomment the following tasks under variant ubuntu1804-debug-asan-all-feature-flags:

  • name: replica_sets_large_txns_format_gen
  • name: replica_sets_large_txns_format_jscore_passthrough
  • name: .replica_sets !.multi_oplog
  • name: .auth
  • name: .encrypt


 Comments   
Comment by Githook User [ 11/Feb/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-53950: fix ASAN errors when migration recipient is stuck on step down
Branch: master
https://github.com/mongodb/mongo/commit/db4c86fb6f59d81a8b74cff41096b9ee6066fe30

Comment by Andrew Shuvalov (Inactive) [ 02/Feb/21 ]

Here is the root cause of the memory leak:

The lifetime of the recipient primary service holding each instance in the `_instances` collection:

  1. RecipientSyncDataCmd has to handle ConflictingOperationInProgress, in case of the test tenant_migration_conflicting_recipient_sync_data_cmds.js this exception is thrown as part of the test
  2. Exception handler here is invoking `recipientService->releaseInstance()`
  3. The releaseInstance() is removing the instance from the list of `_instances`

Why this that a problem? Let's examine the related logic in the recipient service handling the migration:

  1. After the data sync stage the recipient is wait for the _receivedRecipientForgetMigrationPromise future here
  2. Besides the normal flow when the `_receivedRecipientForgetMigrationPromise` is set when the forget migration is set, this promise is also set in _interrupt() here
  3. So the continuation of the sequence and the eventual resource cleanup depends either on normal `forget migration` command or interrupt

In our case we terminate the test and do `step down` without issuing `forget migration`, which is still legal because step down may happen before the migration is complete. So what happens with the interrupt?

  1. On `step down` the interrupt is set to all active instances stored in _instances here
  2. However as we found above the instance that was removed by `releaseInstance()` from `_instances` collection will not receive the interrupt, thus it will continue waiting for the `_receivedRecipientForgetMigrationPromise` promise that is set by the interrupt

The fix that I will try is to simply send `interrupt()` to the instance from `service->releaseInstance()` method. Also need to be careful about deadlocks and to not set the promises in the interrupt handler twice. This seems logical to me because the `releaseInstance()` is removing the instance from interrupt on step down receivers and hence should receive the interrupt then.

Comment by Andrew Shuvalov (Inactive) [ 30/Jan/21 ]

In general the problem is that while 2 recipient services were started by the test jstests/replsets/tenant_migration_conflicting_recipient_sync_data_cmds.js:

[js_test:tenant_migration_conflicting_recipient_sync_data_cmds] 2021-01-29T00:31:47.500+0000 d20770| 2021-01-29T00:31:47.500+00:00 I  REPL     4879607 [TenantMigrationRecipientService-1] "Starting tenant migration recipient instance: ","attr":{"migrationId":{"uuid":{"$uuid":"70af1403-4104-4796-8be6-f65b55ca42af"}},"tenantId":"test","connectionString":"foo/bar:12345","readPreference":"{ mode: \"primary\" }"}

[js_test:tenant_migration_conflicting_recipient_sync_data_cmds] 2021-01-29T00:31:47.501+0000 d20770| 2021-01-29T00:31:47.501+00:00 I REPL 4879607 [TenantMigrationRecipientService-0] "Starting tenant migration recipient instance: ","attr":{"migrationId":{"uuid":{"$uuid":"96538e3b-8e69-43d4-9ccc-c063738592c3"}},"tenantId":"test","connectionString":"foo/bar:12345","readPreference":"{ mode: \"primary\" }"}

Only one of them completed:

[js_test:tenant_migration_conflicting_recipient_sync_data_cmds] 2021-01-29T00:31:48.189+0000 d20770| 2021-01-29T00:31:48.188+00:00 I  REPL     4881402 [TenantMigrationRecipientService-1] "Migration not marked to be garbage collectable","attr":{"migrationId":{"uuid":{"$uuid":"70af1403-4104-4796-8be6-f65b55ca42af"}},"tenantId":"test","status":{"code":11602,"codeName":"InterruptedDueToReplStateChange","errmsg":"PrimaryOnlyService interrupted due to stepdown"}}

Comment by Andrew Shuvalov (Inactive) [ 29/Jan/21 ]

maxh  said it could be related to shutdown timeout:
https://github.com/mongodb/mongo/blob/8942d990feaadd0abeb1140de16768011a649974/src/mongo/transport/service_entry_point_impl.cpp#L240-L244 is where we try harder for things to be shut down under ASan. the 10 second value comes from https://github.com/mongodb/mongo/blob/8942d990feaadd0abeb1140de16768011a649974/src/mongo/db/mongod_main.cpp#L1280 "
 

Comment by Andrew Shuvalov (Inactive) [ 29/Jan/21 ]

Reproduced leak with lingzhi.deng test: https://logkeeper.mongodb.org/lobster/build/a3f568026ccf14fda8bf2aeace035c18/test/60135770be07c42c0721a9a6#bookmarks=0%2C2782&l=1 

Comment by Andrew Shuvalov (Inactive) [ 28/Jan/21 ]

Another ASAN failure lingzhi.deng provided: https://logkeeper.mongodb.org/lobster/build/acb07adeadd1b901a74b40a136d834e6/test/6011c785c2ab68553c9374a2#bookmarks=0%2C4352%2C4353%2C4510%2C4511%2C4564%2C4578%2C4579%2C4580%2C4590%2C4591%2C6073&f~=100~d20020&l=1

[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #9 0x56069d9ae32b in mongo::StatusWith<mongo::executor::TaskExecutor::CallbackHandle> mongo::executor::ScopedTaskExecutor::Impl::_wrapCallback<mongo::executor::ScopedTaskExecutor::Impl::scheduleWork(mongo::unique_function<void (mongo::executor::TaskExecutor::CallbackArgs const&)>&&)::'lambda'(auto&&), mongo::unique_function<void (mongo::executor::TaskExecutor::CallbackArgs const&)> >(auto&&, mongo::unique_function<void (mongo::executor::TaskExecutor::CallbackArgs const&)>&&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/executor/scoped_task_executor.cpp:266
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #10 0x56069d9a685d in mongo::executor::ScopedTaskExecutor::Impl::scheduleWork(mongo::unique_function<void (mongo::executor::TaskExecutor::CallbackArgs const&)>&&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/executor/scoped_task_executor.cpp:139:16
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #11 0x5606a1b562ee in mongo::executor::TaskExecutor::schedule(mongo::unique_function<void (mongo::Status)>) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/executor/task_executor.cpp:115:31
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #12 0x56069979783b in mongo::Future<mongo::future_details::UnwrappedTypeImpl<decltype(fp(std::forward<decltype(fp)>(fp)...))>::type> auto mongo::ExecutorFuture<void>::wrapCBHelper<void ()>(mongo::unique_function<void ()>&&)::'lambda'(auto&&...)::operator()<>(auto&&...) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future.h:1209:15
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #13 0x5606997a0c2b in auto mongo::future_details::call<auto mongo::ExecutorFuture<void>::wrapCBHelper<void ()>(mongo::unique_function<void ()>&&)::'lambda'(auto&&...)&>(void (&&)(), mongo::future_details::FakeVoid) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:255:12
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #14 0x5606997a0c2b in auto mongo::future_details::throwingCall<auto mongo::ExecutorFuture<void>::wrapCBHelper<void ()>(mongo::unique_function<void ()>&&)::'lambda'(auto&&...)&, mongo::future_details::FakeVoid>(void (&&)(), mongo::future_details::FakeVoid&&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:308
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #15 0x5606997a0c2b in auto mongo::future_details::FutureImpl<mongo::future_details::FakeVoid>::then<auto mongo::ExecutorFuture<void>::wrapCBHelper<void ()>(mongo::unique_function<void ()>&&)::'lambda'(auto&&...)>(void (&&)()) &&::'lambda'()::operator()() const::'lambda'(mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*, mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*)::operator()(mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*, mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:935
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #16 0x5606996e8f65 in mongo::unique_function<void (mongo::future_details::SharedStateBase*)>::operator()(mongo::future_details::SharedStateBase*) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/functional.h:109:22
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #17 0x5606996e8f65 in mongo::future_details::SharedStateBase::transitionToFinished() /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:441
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.700+0000 d20020|     #18 0x560699799f27 in mongo::future_details::FutureImpl<mongo::future_details::FakeVoid>::propagateResultTo(mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*) &&::'lambda'()::operator()() const::'lambda'(mongo::future_details::SharedStateBase*)::operator()(mongo::future_details::SharedStateBase*) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:1167:29
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.701+0000 d20020|     #19 0x560699799f27 in void mongo::unique_function<void (mongo::future_details::SharedStateBase*)>::callRegularVoid<mongo::future_details::FutureImpl<mongo::future_details::FakeVoid>::propagateResultTo(mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*) &&::'lambda'()::operator()() const::'lambda'(mongo::future_details::SharedStateBase*)>(std::integral_constant<bool, true>, mongo::future_details::FutureImpl<mongo::future_details::FakeVoid>::propagateResultTo(mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>*) &&::'lambda'()::operator()() const::'lambda'(mongo::future_details::SharedStateBase*)&, mongo::future_details::SharedStateBase*&&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/functional.h:145
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.701+0000 d20020|     #20 0x5606996e8f65 in mongo::unique_function<void (mongo::future_details::SharedStateBase*)>::operator()(mongo::future_details::SharedStateBase*) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/functional.h:109:22
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #21 0x5606996e8f65 in mongo::future_details::SharedStateBase::transitionToFinished() /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:441
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #22 0x5606996e766c in mongo::future_details::SharedStateImpl<mongo::future_details::FakeVoid>::fillChildren(std::forward_list<boost::intrusive_ptr<mongo::future_details::SharedStateBase>, std::allocator<boost::intrusive_ptr<mongo::future_details::SharedStateBase> > > const&) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:615:61
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #23 0x5606996e910f in mongo::future_details::SharedStateBase::transitionToFinished() /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/util/future_impl.h:456:17
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #24 0x56069c73dea4 in mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/db/repl/wait_for_majority_service.cpp:176:42
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #25 0x56069c7406ae in mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0::operator()() const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/db/repl/wait_for_majority_service.cpp:69:50
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #26 0x56069c7406ae in void std::__invoke_impl<void, mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0>(std::__invoke_other, mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:60
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #27 0x56069c7406ae in std::__invoke_result<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0>::type std::__invoke<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:95
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #28 0x56069c7406ae in decltype(auto) std::__apply_impl<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, std::tuple<> >(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0&&, std::tuple<>&&, std::integer_sequence<unsigned long>) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/tuple:1678
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #29 0x56069c7406ae in decltype(auto) std::apply<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, std::tuple<> >(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0&&, std::tuple<>&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/tuple:1687
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #30 0x56069c7406ae in mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()::operator()() /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/stdx/thread.h:186
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #31 0x56069c7406ae in mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0 std::__invoke_impl<void, mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()>(std::__invoke_other, mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:60
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #32 0x56069c7406ae in std::__invoke_result<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0>::type std::__invoke<mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:95
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #33 0x56069c7406ae in decltype(std::__invoke(_S_declval<0ul>())) std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:234
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #34 0x56069c7406ae in std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()> >::operator()() /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:243
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #35 0x56069c7406ae in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0, 0>(mongo::WaitForMajorityService::setUp(mongo::ServiceContext*)::$_0)::'lambda'()> > >::_M_run() /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:186
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|     #36 0x5606a345d63e in execute_native_thread_routine /data/mci/1249d1e2d062c6707fd15627889d42ee/toolchain-builder/tmp/build-gcc-v3.sh-yMM/build/x86_64-mongodb-linux/libstdc++-v3/src/c++11/../../../../../src/combined/libstdc++-v3/src/c++11/thread.cc:80:18
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.702+0000 d20020|

Comment by Andrew Shuvalov (Inactive) [ 26/Jan/21 ]

lingzhi.deng gave an ASAN stack, might or may not be related:

#0 0x00007f3030b32d77 in OPENSSL_LH_strhash () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#1 0x00007f3030b3f580 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#2 0x00007f3030b32cd1 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#3 0x00007f3030b332fa in OPENSSL_LH_retrieve () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#4 0x00007f3030b402b6 in OBJ_sn2nid () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#5 0x00007f3030b40356 in OBJ_txt2obj () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#6 0x00007f3030b4049d in OBJ_txt2nid () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#7 0x00007f30192d9e02 in mongo::x509OidToShortName[abi:cxx11](mongo::StringData) (name=...) at src/mongo/util/net/ssl_manager.cpp:464
#8 0x00007f30192daf99 in mongo::operator<< (os=..., name=...) at src/mongo/util/net/ssl_manager.cpp:697
#9 0x00007f30192db181 in mongo::SSLX509Name::toString[abi:cxx11]() const (this=0x7f300af58e38) at src/mongo/util/net/ssl_manager.cpp:707
#10 0x00007f301a918633 in mongo::DBClientBase::_auth (this=0x7f2feeb76b20, params=...) at src/mongo/client/dbclient_base.cpp:510
#11 0x00007f3025b881bc in mongo::DBClientConnection::_auth (this=0x7f2feeb76b20, params=...) at src/mongo/client/dbclient_connection.cpp:264
#12 0x00007f301a918eda in mongo::DBClientBase::auth (this=0x7f2feeb76b20, params=...) at src/mongo/client/dbclient_base.cpp:563
#13 0x00007f302ddd20ec in mongo::repl::TenantMigrationRecipientService::Instance::_connectAndAuth (this=0x7f2fdf9ca030, serverAddress=..., applicationName=...,
transientSSLParams=0x7f2fdf795630) at src/mongo/db/repl/tenant_migration_recipient_service.cpp:313
#14 0x00007f302ddd27bd in mongo::repl::TenantMigrationRecipientService::Instance::<lambda(const mongo::HostAndPort&)>::operator()(const mongo::HostAndPort &) const (
__closure=0x7f2fe1ba1168, serverAddress=...) at src/mongo/db/repl/tenant_migration_recipient_service.cpp:381

Generated at Thu Feb 08 05:32:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.