[SERVER-31767] Provide a window of snapshot history that is accessible for PIT reads Created: 30/Oct/17 Updated: 30/Oct/23 Resolved: 25/May/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.1 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Michael Cahill (Inactive) | Assignee: | Dianna Hohensee (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | todo_in_code | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||
| Sprint: | Storage NYC 2018-04-09, Storage NYC 2018-04-23, Storage NYC 2018-05-07, Storage NYC 2018-05-21, Storage NYC 2018-06-04 | ||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||
| Description |
|
In 3.6, we permit reads back to the majority commit point (approximately: a little additional slack is currently built in). Attempts to read at an older point will fail. For Global Point in Time Reads, a mongos will need to establish a common timestamp across all mongod nodes involved in a query. In order to do that, it would be helpful for each node to store some additional history. WiredTiger could monitor when it is under cache pressure from storing history, and update the oldest_timestamp in response, with the goal of allowing reads older than the majority commit point where possible without impacting performance. We should also consider whether to make the "comfort level" configurable, to maintain some history even on memory constrained nodes to increase the chance of finding a common point in time across a cluster. |
| Comments |
| Comment by Githook User [ 25/May/18 ] | |||||||||||||||||||||||||||||||||||
|
Author: {'username': 'DiannaHohensee', 'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com'}Message: | |||||||||||||||||||||||||||||||||||
| Comment by Dianna Hohensee (Inactive) [ 26/Apr/18 ] | |||||||||||||||||||||||||||||||||||
|
bruce.lucas This is the layout I'm planning to add the serverStatus.wiredtiger output, per our conversation. If you have any comments for clarity or requests, let me know.
| |||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 29/Mar/18 ] | |||||||||||||||||||||||||||||||||||
|
I believe you can retrieve the relevant information from WiredTiger via querying an existing statistic. I've written a possible implementation:
The lookaside score is a value between 0 and 100, if there is no cache pressure due to history then the statistic will return a value of 0, as the amount of cache pressure due to history increases the score will increase towards the cap of 100. The above code suggests using a value of 50 as a reasonable threshold to base the decision on - I believe that's right, but don't have concrete data to back it up. | |||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 13/Mar/18 ] | |||||||||||||||||||||||||||||||||||
|
The majority of the work here should be done in | |||||||||||||||||||||||||||||||||||
| Comment by Ian Whalen (Inactive) [ 12/Mar/18 ] | |||||||||||||||||||||||||||||||||||
|
alexander.gorrod sounds like there is a dependency on some WT work to produce a callback when there is cache pressure - can you either link that ticket here or file and link if it doesn't exist yet? |