Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38299

killOp while preparing a transaction can trigger std::terminate

    XMLWordPrintable

    Details

    • Operating System:
      ALL
    • Sprint:
      Sharding 2018-12-17, Sharding 2018-12-31, Sharding 2019-01-14, Sharding 2019-01-28, Sharding 2019-02-11

      Description

      Similar to SERVER-38190, if killOp kills a thread preparing a transaction it can trigger a ScopeGuard which will try to abort the active transaction, which will throw again because it re-uses the killed operation context, triggering a std::terminate.

      Example failure:

      [ShardedClusterFixture:job0:shard1:primary] 2018-11-28T14:36:16.007-0500 F STORAGE  [conn66] Caught exception during abort of prepared transaction 3 on { id: UUID("1e2a467a-b07c-497a-bdcf-49009e370802"), uid: BinData(0, E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855) }: Interrupted: operation was interrupted
      [ShardedClusterFixture:job0:shard1:primary] 2018-11-28T14:36:16.007-0500 F -        [conn66] terminate() called. An exception is active; attempting to gather more information
      [ShardedClusterFixture:job0:shard1:primary] 2018-11-28T14:36:16.009-0500 F -        [conn66] DBException::toString(): Interrupted: operation was interrupted
      [ShardedClusterFixture:job0:shard1:primary] Actual exception type: mongo::error_details::ExceptionForImpl<(mongo::ErrorCodes::Error)11601, mongo::ExceptionForCat<(mongo::ErrorCategory)1> >
      [ShardedClusterFixture:job0:shard1:primary]  0x7f22f4d313e3 0x7f22f4d3061c 0x7f22fd83a816 0x7f22fd83a861 0x7f22f8c56dd0 0x7f22f8c56b99 0x7f22f8c56b42 0x7f22f8c4bc99 0x7f22f8c4ba31 0x7f22fd43424d 0x7f22fd433a98 0x7f22fd433982 0x7f22fd433655 0x7f22f1c18496 0x7f22f1c175a4 0x7f22f1c15422 0x7f22f1c121c6 0x7f22f1c0f20d 0x7f22f1c0deb3 0x7f22fc36d775 0x7f22fc333d9c 0x7f22fc32e6d0 0x7f22fc332c78 0x7f22fc332a7d 0x7f22fe0fb3ce 0x7f22fc2d86e7 0x7f22fc32e226 0x7f22fc32d9ba 0x7f22fc330918 0x7f22fc3307df 0x7f22fc3305f3 0x7f22fc33039a 0x7f22fc32d4dd 0x7f22fc32d294 0x7f22fc32e653 0x7f22fc332c78 0x7f22fc332a7d 0x7f22fe0fb3ce 0x7f22fc2d914c 0x7f22fc2d8ebd 0x7f22fe0fb3ce 0x7f22f501a09c 0x7f22f26b76db 0x7f22f23e088f
      [ShardedClusterFixture:job0:shard1:primary] ----- BEGIN BACKTRACE -----
      [ShardedClusterFixture:job0:shard1:primary]  libbase.so(_ZN5mongo15printStackTraceERSo+0x33) [0x7f22f4d313e3]
      [ShardedClusterFixture:job0:shard1:primary]  libbase.so(+0x27761C) [0x7f22f4d3061c]
      [ShardedClusterFixture:job0:shard1:primary]  libjournal_listener.so(_ZN10__cxxabiv111__terminateEPFvvE+0x6) [0x7f22fd83a816]
      [ShardedClusterFixture:job0:shard1:primary]  libjournal_listener.so(+0xF2861) [0x7f22fd83a861]
      [ShardedClusterFixture:job0:shard1:primary]  libcatalog_raii.so(+0xDADD0) [0x7f22f8c56dd0]
      [ShardedClusterFixture:job0:shard1:primary]  libcatalog_raii.so(+0xDAB99) [0x7f22f8c56b99]
      [ShardedClusterFixture:job0:shard1:primary]  libcatalog_raii.so(+0xDAB42) [0x7f22f8c56b42]
      [ShardedClusterFixture:job0:shard1:primary]  libcatalog_raii.so(+0xCFC99) [0x7f22f8c4bc99]
      [ShardedClusterFixture:job0:shard1:primary]  libcatalog_raii.so(_ZN5mongo22TransactionParticipant18prepareTransactionEPNS_16OperationContextEN5boost8optionalINS_4repl6OpTimeEEE+0x781) [0x7f22f8c4ba31]
      [ShardedClusterFixture:job0:shard1:primary]  libcommands_db_s.so(+0x10524D) [0x7f22fd43424d]
      [ShardedClusterFixture:job0:shard1:primary]  libcommands_db_s.so(+0x104A98) [0x7f22fd433a98]
      [ShardedClusterFixture:job0:shard1:primary]  libcommands_db_s.so(+0x104982) [0x7f22fd433982]
      [ShardedClusterFixture:job0:shard1:primary]  libcommands_db_s.so(+0x104655) [0x7f22fd433655]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(+0x49496) [0x7f22f1c18496]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(+0x485A4) [0x7f22f1c175a4]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(+0x46422) [0x7f22f1c15422]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(+0x431C6) [0x7f22f1c121c6]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(+0x4020D) [0x7f22f1c0f20d]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point_common.so(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x523) [0x7f22f1c0deb3]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_context_d.so(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x65) [0x7f22fc36d775]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0x26C) [0x7f22fc333d9c]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x1F0) [0x7f22fc32e6d0]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x43C78) [0x7f22fc332c78]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x43A7D) [0x7f22fc332a7d]
      [ShardedClusterFixture:job0:shard1:primary]  libstorage_ephemeral_for_test_core.so(_ZNKSt8functionIFvvEEclEv+0x3E) [0x7f22fe0fb3ce]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_executor.so(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x167) [0x7f22fc2d86e7]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0xE6) [0x7f22fc32e226]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0x18A) [0x7f22fc32d9ba]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x41918) [0x7f22fc330918]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x417DF) [0x7f22fc3307df]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x415F3) [0x7f22fc3305f3]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x4139A) [0x7f22fc33039a]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x3E4DD) [0x7f22fc32d4dd]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0xE4) [0x7f22fc32d294]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x173) [0x7f22fc32e653]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x43C78) [0x7f22fc332c78]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_entry_point.so(+0x43A7D) [0x7f22fc332a7d]
      [ShardedClusterFixture:job0:shard1:primary]  libstorage_ephemeral_for_test_core.so(_ZNKSt8functionIFvvEEclEv+0x3E) [0x7f22fe0fb3ce]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_executor.so(+0x4314C) [0x7f22fc2d914c]
      [ShardedClusterFixture:job0:shard1:primary]  libservice_executor.so(+0x42EBD) [0x7f22fc2d8ebd]
      [ShardedClusterFixture:job0:shard1:primary]  libstorage_ephemeral_for_test_core.so(_ZNKSt8functionIFvvEEclEv+0x3E) [0x7f22fe0fb3ce]
      [ShardedClusterFixture:job0:shard1:primary]  libtransport_layer_common.so(+0xA09C) [0x7f22f501a09c]
      [ShardedClusterFixture:job0:shard1:primary]  libpthread.so.0(+0x76DB) [0x7f22f26b76db]
      [ShardedClusterFixture:job0:shard1:primary]  libc.so.6(clone+0x3F) [0x7f22f23e088f]
      [ShardedClusterFixture:job0:shard1:primary] -----  END BACKTRACE  -----
      

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: