[SERVER-37189] transactions.currentActive has a value of -1 Created: 18/Sep/18  Updated: 29/Oct/23  Resolved: 15/Nov/18

Status: Closed
Project: Core Server
Component/s: Diagnostics, Replication
Affects Version/s: 4.0.2, 4.1.2
Fix Version/s: 4.0.6, 4.1.6

Type: Bug Priority: Major - P3
Reporter: Isabella Siu (Inactive) Assignee: Tess Avitabile (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File txns_secondary_metrics.js    
Issue Links:
Backports
Related
related to SERVER-33580 Restrict multi-statement transactions... Closed
related to SERVER-34557 Allow running transactions against pr... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0
Steps To Reproduce:

on a secondary:

sessionOptions = {causalConsistency: false}
sess = db.getMongo().startSession(sessionOptions)
sess.startTransaction()
db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 })
db.abc.insert({_id:213})
sess.getDatabase("test").abc.insert({_id: "insert-4"}) // this had a "TransientTransactionError"
db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 })
sess.getDatabase("test").abc.insert({_id: "insert-4"}) // this had a "TransientTransactionError"
db.serverStatus({ repl: 0, metrics: 0, locks: 0, wiredTiger: 0 })

Sprint: Repl 2018-10-22, Repl 2018-11-05, Repl 2018-11-19
Participants:
Linked BF Score: 0

 Description   

Got a value of -1 for the currentActive transactions when running serverStatus against a replica set secondary.

"transactions" : {
 "retriedCommandsCount" : NumberLong(0),
 "retriedStatementsCount" : NumberLong(0),
 "transactionsCollectionWriteCount" : NumberLong(0),
 "currentActive" : NumberLong(-1),
 "currentInactive" : NumberLong(1),
 "currentOpen" : NumberLong(0),
 "totalAborted" : NumberLong(1),
 "totalCommitted" : NumberLong(0),
 "totalStarted" : NumberLong(1)
}



 Comments   
Comment by Githook User [ 21/Dec/18 ]

Author:

{'username': 'tessavitabile', 'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile'}

Message: SERVER-37189 Record metrics correctly for transactions that fail prior to unstash

(cherry picked from commit 06dd323b423deccb5fbb650ddbed0135c00191d2)
Branch: v4.0
https://github.com/mongodb/mongo/commit/0bf668f3336ce9fd7da430a788405fca47158416

Comment by Githook User [ 15/Nov/18 ]

Author:

{'name': 'Tess Avitabile', 'email': 'tess.avitabile@mongodb.com', 'username': 'tessavitabile'}

Message: SERVER-37189 Record metrics correctly for transactions that fail prior to unstash
Branch: master
https://github.com/mongodb/mongo/commit/06dd323b423deccb5fbb650ddbed0135c00191d2

Comment by William Schultz (Inactive) [ 19/Sep/18 ]

Yep, looks like there is an issue here with how we update internal transaction metadata when the first command of a transaction fails against a secondary. On secondaries, we don't actually fail the transaction until the first command is executed. Before this, we treat the transaction as if it was started inside TransactionParticipant::_beginMultiDocumentTransaction. This would increase the currentActive count to 1. When the command invocation fails with NotMaster, we don't actually throw an exception, but return an ok:0 response. This means that we don't try to abort the transaction here. So, even though the transaction appears to fave "failed" to begin, the internal transaction metadata indicates that there is a transaction in-progress, and the metrics counters reflect that. I need to investigate a bit more to determine the solution to this, but we may need a better way of failing transactions early if they are run against secondaries. I attached an additional repro that demonstrates the issue, against master branch as well.

Comment by Isabella Siu (Inactive) [ 19/Sep/18 ]

william.schultz I do not have test commands enabled, and it does not occur if it is run against a primary. 

Comment by Bruce Lucas (Inactive) [ 19/Sep/18 ]

Working from her repro,

  • problem occurs in 4.0.2 as well
  • does only occur on secondary
  • --setParameter enableTestCommands=1 does not seem to make a difference

Note that even though this only occurs on a secondary where transactions are not allowed, if that secondary were then promoted to primary I assume the accounting error would persist.

Comment by William Schultz (Inactive) [ 18/Sep/18 ]

isabella.siu In 4.0, we explicitly disallow transactions on replica set secondaries. See SERVER-33580 and SERVER-34557. Does this issue still occur if the transaction is run against a primary?

Note: Transactions will be allowed to run against secondaries if test commands are enabled, so it might be worth checking that in your setup.

Comment by Isabella Siu (Inactive) [ 18/Sep/18 ]

william.schultz

Generated at Thu Feb 08 04:45:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.