Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36921

Transaction lock timeout errors when running transactions concurrently with Logical Session Cache refreshes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 4.1.4
    • None
    • None
    • Sharding
    • Fully Compatible
    • ALL

    Description

      TransientTransactionErrors were uncovered on multiple tests when running a custom jsCore passthrough suite with the Logical Session Cache refresh set to 100ms.

      The cleanest example of this error is in multi_statement_transaction – but there aren't any log lines in this test to investigate.

      assert: command failed: {
      		 
      errorLabels" : [
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.039+0000 		"TransientTransactionError"
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.039+0000 	],
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.039+0000 	"operationTime" : Timestamp(1535488481, 2),
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.039+0000 	"ok" : 0,
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.043+0000 	"errmsg" : "Unable to acquire lock '{6237343057549539649: Database, 1625657039122151745, config}' within a max lock request timeout of '5ms' milliseconds.",
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.043+0000 	"code" : 24,
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.043+0000 	"codeName" : "LockTimeout",
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.050+0000 	"$clusterTime" : {
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.051+0000 		"clusterTime" : Timestamp(1535488481, 2),
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.051+0000 		"signature" : {
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.052+0000 			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.052+0000 			"keyId" : NumberLong(0)
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.052+0000 		}
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.052+0000 	}
      		 
      [js_test:multi_statement_transaction] 2018-08-28T20:34:41.053+0000 }

      The error also persists in the collation and transactions_profiling_with_drops tests (which have plenty of logs to look at).

      After a cursory investigation of collation.js: it seems that a connection takes an exclusive write lock on config.system.sessions for an attempted update, then the LogicalSessionCache thread tries to take the lock as well – then fails. Check the highlighted lines on this Lobster log.

      Attachments

        Issue Links

          is caused by

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-36961 createIndexes command should check if index already exists with weak lock

          • Critical - P2 - Crashes, loss of data, severe memory leak.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-37511 Logical session reaper and refresh threads should set up sessions collection immediately

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              backlog-server-sharding Backlog - Sharding Team
              blake.oler@mongodb.com Blake Oler
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: