[SERVER-40337] Remove oldestOpenUnpreparedReadTimestamp Created: 26/Mar/19  Updated: 29/Oct/23  Resolved: 09/Apr/19

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: None
Fix Version/s: 4.1.10

Type: Bug Priority: Major - P3
Reporter: Tess Avitabile (Inactive) Assignee: Lingzhi Deng
Resolution: Fixed Votes: 0
Labels: prepare_diagnostics
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-12628 Docs for SERVER-40337: Remove oldestO... Closed
Related
related to WT-4704 Add statistic tracking oldest active ... Closed
is related to SERVER-35449 Return the oldest read timestamp used... Closed
is related to SERVER-35713 Add oldest open transaction read time... Closed
is related to WT-4154 Surface the oldest read timestamp Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Repl 2019-04-08, Repl 2019-04-22
Participants:

 Description   

The serverStatus metric oldestOpenUnpreparedReadTimestamp is named to represent the oldest read timestamp of any unprepared transaction. However, as described in WT-4154, this value is the oldest_active_reader, which is the minimum read timestamp of all active readers. Timestamped reads can occur without multi-document transactions, e.g. all secondary reads, so this value exists even when the server has no open, unprepared transactions. We will remove this serverStatus metrics, and the oldest active read timestamp will be added to WT statistics in WT-4704.



 Comments   
Comment by Lingzhi Deng [ 09/Apr/19 ]

Documentation changes: remove transactions.oldestOpenUnpreparedReadTimestamp from serverStatus.

Comment by Githook User [ 09/Apr/19 ]

Author:

{'email': 'lingzhi.deng@mongodb.com', 'name': 'Lingzhi Deng', 'username': 'ldennis'}

Message: SERVER-40337: Remove oldestOpenUnpreparedReadTimestamp from serverStatus.transactions

This reverts most of the work done in 3bb72042c29ceac96ec628ee643cbc4db8e982ab
Branch: master
https://github.com/mongodb/mongo/commit/decf670a6d7ec7a31c3820032188aab3fdc0fbd1

Comment by Tess Avitabile (Inactive) [ 05/Apr/19 ]

Wouldn't the oldest reader identify history being pinned by any query, not by a multi-document transaction read per se? Or will we use the oldest reader value to determine that it's not the cause of pinned history because it's later than the current oldest timestamp value?

Yes, the oldest active read timestamp will identify history being pinned by any query. However, for timestamped reads other than multi-document transactions (e.g. majority reads and secondary reads), I would expect them to seldom be the cause of pinned history because they yield their snapshot. The oldest active read timestamp might be helpful in identifying whether transactions that run close to the 1 minute limit are problematic. Or, as you said, we could use the oldest active read timestamp to determine that readers are not the cause of pinned history.

Comment by Eric Milkie [ 05/Apr/19 ]

Wouldn't the oldest reader identify history being pinned by any query, not by a multi-document transaction read per se? Or will we use the oldest reader value to determine that it's not the cause of pinned history because it's later than the current oldest timestamp value?

Comment by Tess Avitabile (Inactive) [ 04/Apr/19 ]

I think reporting the oldest reader is independently useful, since this can be used to diagnose history being pinned by transactions. That was our original intention with reporting oldestOpenUnpreparedReadTimestamp.

Comment by Alexander Gorrod [ 04/Apr/19 ]

milkie We already report transaction range of timestamps currently pinned - which reports the lower of either oldest_timestamp and the oldest reader. If reporting the oldest reader as a statistic is independently useful I'm happy to add it.

Comment by Eric Milkie [ 04/Apr/19 ]

Hi alexander.gorrod, I was wondering if it would be possible to add an oldest_active_reader as a true WiredTiger statistic that would be returned alongside all the other stats returned by statistics=(fast), or if the calculation of this particular stat would not qualify as "fast".
If we could have WiredTiger output this stat itself, then we wouldn't need to separately query it and output it in serverStatus in the MongoDB code.

Comment by Tess Avitabile (Inactive) [ 04/Apr/19 ]

Good point, I see that this is part of the general StorageInterface. Do you think it should go in the storageEngine section? My only reservation about putting it there was that storageEngine seems to mostly contain unchanging properties of the storage engine. However, maybe that's the best place for it.

Comment by Eric Milkie [ 03/Apr/19 ]

I understand the naming change, but I don't understand why we would move this into the wiredTiger section. Is there an assumption that other storage engines that support timestamps won't have such a value?

Comment by Lingzhi Deng [ 03/Apr/19 ]

We want to move the oldest active reader timestamp (i.e. query_timestamp(‘get=oldest_reader’)) to the wiredTiger serverStatus. But I am not quite sure which section under wiredTiger should I put that into. I was thinking wiredTiger.transaction but it doesnt seem quite right. milkie, geert.bosch, what's your take on exposing the timestamp to wiredTiger serverStatus? Thanks.

Comment by Tess Avitabile (Inactive) [ 27/Mar/19 ]

Thanks, alexander.gorrod, I'm glad there is diagnosability for this.

Comment by Alexander Gorrod [ 27/Mar/19 ]

tess.avitabile if there is no read timestamp, cache pressure is caused by keeping transaction IDs active. Those are captured in WiredTiger statistics:

transaction range of IDs currently pinned by a checkpoint
transaction range of IDs currently pinned
transaction range of timestamps currently pinned
transaction range of timestamps pinned by a checkpoint
transaction range of timestamps pinned by the oldest timestamp

There is no way to correlate transaction IDs to elapsed time, but a large or growing range in transaction range of IDs currently pinned should give you the relevant information.

Comment by Tess Avitabile (Inactive) [ 26/Mar/19 ]

Another concern is that after SERVER-38906, only transactions with readConcern level snapshot perform timestamped reads. This means that this serverStatus metric will not capture history pinned by all multi-document transactions. Is there a way to capture this potential source of cache pressure? alexander.gorrod

Generated at Thu Feb 08 04:54:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.