[SERVER-40105] Improve diagnostic information in currentOp for excessive write conflicts and prepare conflicts Created: 13/Mar/19  Updated: 29/Oct/23  Resolved: 12/Apr/19

Status: Closed
Project: Core Server
Component/s: Concurrency, Storage
Affects Version/s: None
Fix Version/s: 4.1.11

Type: Question Priority: Major - P3
Reporter: Geert Bosch Assignee: Louis Williams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-12634 Docs for SERVER-40105: Improve diagno... Closed
Related
related to SERVER-79123 Tests enabling skipWriteConflictRetri... Closed
is related to SERVER-40062 Add failpoint to skip doing retries o... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2019-04-08, Storage NYC 2019-04-22
Participants:

 Description   

Currently, when a non-transactional write conflicts with an active transaction, that write may keep retrying thousands of times until the transaction finally commits or aborts. This situation is hard to diagnose, because currentOp doesn't show that the active non-transactional writes have occurred so many retries.

Changes made by this ticket:
The top-level fields 'writeConflicts' and 'prepareReadConflicts' are now present in the currentOp() output.

The number of write conflicts represents the number of times a write attempt conflicted with another operation on the same document.

The number of prepare conflicts represents the number of times the operation blocked waiting for a write by a prepared transaction to commit or abort. An operation blocked on prepare conflicts holds both locks and storage engine resources while waiting.



 Comments   
Comment by Githook User [ 12/Apr/19 ]

Author:

{'name': 'Louis Williams', 'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com'}

Message: SERVER-40105 fix unittests
Branch: master
https://github.com/mongodb/mongo/commit/32da14821b53da78b1316e46905b1babe22dcf74

Comment by Githook User [ 12/Apr/19 ]

Author:

{'name': 'Louis Williams', 'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com'}

Message: SERVER-40105 Improve diagnostic information in currentOp for writeConflicts and prepareConflicts
Branch: master
https://github.com/mongodb/mongo/commit/ee9c8b7d7382cb7f91b72c2e680f18016754ee35

Comment by David Storch [ 05/Apr/19 ]

louis.williams geert.bosch asya, ah, I actually think this is a good reason to not include metrics other then prepare conflicts and write conflicts in $currentOp output. The presence of these metrics in $currentOp implies that they are updated incrementally during the execution of a query. But actually doing so would require further work in the query engine. Perhaps the scope of this ticket should be restored to its original size of only adding prepareReadConflicts and writeConflicts to $currentOp?

Comment by Louis Williams [ 05/Apr/19 ]

asya, in my code review I added a top-level field, "metrics", to currentOp. david.storch wanted me to check with you if we would prefer using a different field name that encompasses Bruce's list above.

With this change the following metrics will update in currentOp as an operation is in progress, whereas before this change they only update at the end of an operation so the in-progress values are never seen in currentOp

bruce.lucas this isn't entirely correct. While these metrics will now have the ability to be observed as an operation progresses, they aren't actually updated as the increase, but when the operation completes. The exceptions to this are prepare conflicts and write conflicts, which are incremented as they happen.

Comment by Louis Williams [ 27/Mar/19 ]

It seems like our diagnostic information in currentOp does not include prepare conflicts either. I'm amending this ticket to go ahead an include all additive metrics from OpDebug in the currentOp output because it would honestly be harder to just include writeConflicts and nothing else.

Generated at Thu Feb 08 04:54:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.