-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines
-
58.995
-
None
-
None
__gen_oldest() linearly scans the entire session array (conn->session_cnt slots) on every call.
Generation checks run on every page/memory free, so discarding a large tree from cache - e.g. __wt_evict_file(WT_SYNC_DISCARD) during a non-forced drop - costs O(in-cache pages * active sessions).
Observed in cetrain cases: perf flame graphs show __gen_oldest dominating CPU during ident drops.
The stall caused connection pile-up, raising open session counts to 3-7k which further slowed the discard - a feedback loop.
Single-ident discards held the schema and dhandle-list write locks for 16-70s.
The discard path amplifies the cost but any generation check under high session counts pays the same linear scan.
Possible directions (to be evaluated):
- Cache/track the oldest generation instead of recomputing by full scan
- Maintain a per-generation minimum incrementally as sessions enter/leave
- Batch frees during file discard so one generation check covers many pages
- related to
-
WT-11921 Investigate inefficient interaction between drop table and sweep sever
-
- Open
-
-
WT-12798 Checkpoint time is high when the cache size is large
-
- Open
-
-
WT-18427 Non-forced drop discards the tree from cache synchronously while holding the schema and dhandle-list write locks
-
- Needs Scheduling
-