[SERVER-46435] Alternatives to using the lookaside-score for snapshot (history) window control [v4.2 only] Created: 26/Feb/20  Updated: 06/Dec/22  Resolved: 01/Feb/21

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on WT-5995 Need a means by which to detect WT ca... Closed
Related
related to WT-5124 Investigate why the WT lookaside scor... Closed
Assigned Teams:
Storage Execution
Sprint: Execution Team 2020-03-23, Execution Team 2020-04-20
Participants:

 Description   

A few potential solutions I can think of:
1) Consider (dirty-bytes-in-cache / total-bytes-in-cache == 0.20 || bytes-in-use-in-cache / total-bytes-in-cache == 0.95) as cache pressure and reduce window size.
2) Make the window size constant and only user adjustable.
3) Make the window have a settable minimum value (new startup+runtime server parameter)



 Comments   
Comment by Connie Chen [ 01/Feb/21 ]

We would be potentially destabilizing 4.2 with this change, closing as won't do. 

Comment by Dianna Hohensee (Inactive) [ 06/May/20 ]

The WT eviction_trigger configuration defaults to 95% and is when application threads start to perform eviction. At that point, performance is starting to suffer / become unreliable.

WT also has an eviction_target configuration that defaults to 80% at which it tries to keep the cache fullness. WT eviction threads are active when this threshold is reached. So WT seems to try to keep the cache relatively full, so that reads are fast, I imagine.

WT also has an eviction_dirty_target (default 5%) and eviction_dirty_trigger (default 20%). These behave similarly and can co-opt application threads for eviction.

I've been looking at the information here http://source.wiredtiger.com/3.2.1/tune_cache.html to conclude the above.

-------------------------------------------------------------------------------

Two potential solutions I can think of:
1) Consider (dirty-bytes-in-cache / total-bytes-in-cache == 0.20 || bytes-in-use-in-cache / total-bytes-in-cache == 0.95) as cache pressure and reduce window size.
2) Make the window size constant and only user adjustable.

Comment by Dianna Hohensee (Inactive) [ 08/Apr/20 ]

Lookaside table inserts are not logically going to work because insert frequency could drop while cache pressure remains high. And it is thought (unconfirmed) that inserts can occur before there's any significant cache pressure.

Luke Pearson suggested these metrics:

Cache bytes max: Total configured cache size
Cache bytes in use: total amount of bytes in use
Cache bytes dirty: Amount of dirty bytes in cache (internal + leaf pages).
Cache bytes internal: Bytes belonging to internal pages in the cache.
Cache bytes leaf: Bytes belonging to leaf pages in the cache.
 
Conn_cache.c __wt_cache_stats_update is a good place to look at where the values are being set.

So I'll look into them.

Comment by Connie Chen [ 27/Feb/20 ]

Ensure to confirm with WT that this is change is correct

Generated at Thu Feb 08 05:11:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.