[SERVER-44870] MongoDB to allow WiredTiger to return WT_ROLLBACK for all transactions Created: 27/Nov/19 Updated: 21/Nov/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Eric Milkie | Assignee: | Backlog - Storage Execution Team |
| Resolution: | Unresolved | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Assigned Teams: |
Storage Execution
|
||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||
| Description |
|
Currently, WT in MongoDB will not return WT_ROLLBACK for transactions that have not yet performed an update. These cases could be desirable for system stability. |
| Comments |
| Comment by Keith Bostic (Inactive) [ 16/Feb/22 ] |
|
FWIW, this is still an issue: BF-24076/ |
| Comment by Brian Lane [ 23/Jul/20 ] |
|
connie.chen - that's fine. Could this perhaps be tagged as a candidate for a QW in a future quarter? |
| Comment by Connie Chen [ 20/Jul/20 ] |
|
brian.lane - is it ok to still leave this in backlog for now? Has this become more urgent? |
| Comment by Brian Lane [ 20/Dec/19 ] |
|
Thanks keith.bostic, I will put this in the backlog for now. |
| Comment by Keith Bostic (Inactive) [ 16/Dec/19 ] |
|
milkie, based on alexander.gorrod's comments in SERVER-44870, I've updated the ticket to broaden its scope: as I now understand it, the storage layer won't return WT_ROLLBACK for any MongoDB transaction that has yet to do an update. I can't speak to how important this is to server operations, but where we see this problem in our test programs is a thread with a read timestamp in the past that hasn't yet done an update, and which has been grabbed to help with eviction. When eviction is having trouble and the read timestamp is a problem, pinning the reader inside the storage eviction loop can just make things worse, of course. Given the broader scope of SERVER-44870, perhaps we should revisit its scheduling (or decide it's more work than originally understood and should wait for a later release).
|
| Comment by Alexander Gorrod [ 11/Dec/19 ] |
I agree - that's how it works. Note that I pulled something out of the middle of the excerpt there.
That's true too, but it's not only applicable to implicit transactions.
All transactions avoid allocating a transaction ID internally until they do an update. When an explicit transaction is begun it will allocate a snapshot if necessary, but it doesn't get a transaction ID until it needs one (which is when it goes to do an update operation). |
| Comment by Louis Williams [ 05/Dec/19 ] |
|
We already have a ticket to assert that normal cursor operations use explicit transactions, because we believe they should: I think we should use this ticket to evaluate whether the current users of getSessionNoTxn can handle WT_ROLLBACK errors. |
| Comment by Keith Bostic (Inactive) [ 05/Dec/19 ] |
|
In talking with louis.williams, I think I've spotted the problem. I've not been clear that I'm talking about implicit transactions. What I believe to be the case is that once a transaction does something that gets it allocated a transaction ID (so, an update/insert, or, being part of an explicit transaction created in the application), then the WiredTiger layer will return WT_ROLLBACK, expecting the caller is able to handle it. If it's an implicit transaction, that has yet to do an update/insert, then the WiredTiger layer won't return WT_ROLLBACK, because we're not sure the MongoDB operation can handle it. This isn't really my area of expertise, so YMMV, but I think that's how it works. |
| Comment by Keith Bostic (Inactive) [ 05/Dec/19 ] |
No, I'm talking about cursors that have not (yet) performed any update operations. |
| Comment by Louis Williams [ 05/Dec/19 ] |
|
I think we're using different terms for "read-only". You're talking about WT "readonly=true" cursors, correct? We don't use those in MongoDB outside --queryableBackupMode. When we talk about reads in MongoDB, I mean that we start a transaction and abort it at the end of the operation without doing any writes. We don't pass any special configuration to WT. In this case, WT_ROLLBACK can be returned, correct? Is there a plan to start using read-only transactions in MongoDB? For all operations, we convert WT_ROLLBACK to a WriteConflictException. In MongoDB, outside of a writeConflictRetry(), read operations will pass this error to clients. Internal threads are definitely not prepared to handle WT_ROLLBACK for reads, but I imagine this is already a problem today. |
| Comment by Keith Bostic (Inactive) [ 05/Dec/19 ] |
I don't believe that's true. The code that raised this question is when an application thread is tapped to help with eviction. In that case, if the transaction is readonly, it will not return WT_ROLLBACK (or any error), even if the transaction is contributing to the problem (for example, it's an older reading pinning content in the cache). I haven't done a review, but in general I think it's the case that the places where WiredTiger returns WT_ROLLBACK are not cases triggered by readonly operations. I don't know any of the MongoDB history on this, alexander.gorrod just told me the upper levels of MongoDB weren't prepared to handle WT_ROLLBACK from readonly operations. |
| Comment by Louis Williams [ 05/Dec/19 ] |
|
My understanding is that WT already returns WT_ROLLBACK for reads when the cache is full. keith.bostic, Eric said you might be able to elaborate on how this is different than what we do today? |