[SERVER-77542] Internal operations should handle TemporarilyUnavailable and TransactionTooLargeForCache exceptions Created: 29/May/23  Updated: 29/Oct/23  Resolved: 01/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0, 7.0.2, 7.1.0-rc2

Type: Bug Priority: Major - P3
Reporter: Josef Ahmad Assignee: Josef Ahmad
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-80461 Handle TemporarilyUnavailable and Tra... Closed
related to SERVER-63620 Evaluate locations that throw WriteCo... Backlog
is related to SERVER-78702 Consolidate WriteConflictException an... Open
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.1
Sprint: Execution EMEA Team 2023-06-26, Execution EMEA Team 2023-07-10, Execution EMEA Team 2023-07-24, Execution EMEA Team 2023-09-04
Participants:
Linked BF Score: 120

 Description   

Unlike user operations which transparently handle both WriteConflictException (WCE) and (TemprorarilyUnavailableException) TUE via the writeConflictRetry helper, internal operations tend to use custom code that handles WCE's only; oplog truncation is an example. Excerpt from an internal stress test:

{"t":{"$date":"2023-05-12T18:06:13.149+00:00"},"s":"F",  "c":"STORAGE",  "id":6761100, "ctx":"OplogCapMaintainerThread-local.oplog.rs","msg":"Error in OplogCapMaintainerThread","attr":{"error":{"code":365,"codeName":"TemporarilyUnavailable","errmsg":"-31800: oldest pinned transaction ID rolled back for eviction"}}}

We should audit the code for missing TUE handling and consider adopting the writeConflictRetry helper or implementing an equivalent one for internal operations.



 Comments   
Comment by Githook User [ 07/Sep/23 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-77542 Handle TemporarilyUnavailable and TransactionTooLargeForCache as write conflicts for internal operations

(cherry picked from commit 2ed9c7d078e499a02f67c8a8199c4d84fe8dedd9)
Branch: v7.0
https://github.com/mongodb/mongo/commit/bed7540a361f88e17d086e1698ca42bd995c9132

Comment by Githook User [ 07/Sep/23 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-77542 Make internal ops handle TemporarilyUnavailable and TransactionTooLargeForCache exceptions

(cherry picked from commit 02d317cf8ab124ec10880d2cd6232035c731e40e)
Branch: v7.0
https://github.com/mongodb/mongo/commit/d8b33e75cb965b5adb9b405198b7837b4757a6b9

Comment by Githook User [ 06/Sep/23 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-77542 Handle TemporarilyUnavailable and TransactionTooLargeForCache as write conflicts for internal operations

(cherry picked from commit 2ed9c7d078e499a02f67c8a8199c4d84fe8dedd9)
Branch: v7.1
https://github.com/mongodb/mongo/commit/c90c98cb81a0e1d2f8609c3e60f4db833e099433

Comment by Githook User [ 01/Sep/23 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-77542 Handle TemporarilyUnavailable and TransactionTooLargeForCache as write conflicts for internal operations
Branch: master
https://github.com/mongodb/mongo/commit/2ed9c7d078e499a02f67c8a8199c4d84fe8dedd9

Comment by Josef Ahmad [ 07/Aug/23 ]

Reopening due to an issue with converting the TemporarilyUnavailableException into a WriteConflictException. We should handle the TUE as a WCE instead of throwing a WCE, otherwise the write conflict escapes to the caller.

Comment by Josef Ahmad [ 19/Jul/23 ]

This ticket introduces a base StorageUnavailableException encompassing WriteConflictException, TemporarilyUnavailableException, and TransactionTooLargeForCacheException. This base exception is used to catch the three derived exceptions for internal operations where the standard policy implemented by the writeConflictRetry helper does not apply. This is an internal change only; there are no externally-visible changes.

Comment by Githook User [ 18/Jul/23 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-77542 Make internal ops handle TemporarilyUnavailable and TransactionTooLargeForCache exceptions
Branch: master
https://github.com/mongodb/mongo/commit/02d317cf8ab124ec10880d2cd6232035c731e40e

Generated at Thu Feb 08 06:35:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.