__gen_oldest scans the full session array per call making large tree discards O(pages*sessions)

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines
    • 58.995
    • None
    • None

      __gen_oldest() linearly scans the entire session array (conn->session_cnt slots) on every call.
      Generation checks run on every page/memory free, so discarding a large tree from cache - e.g. __wt_evict_file(WT_SYNC_DISCARD) during a non-forced drop - costs O(in-cache pages * active sessions).

      Observed in cetrain cases: perf flame graphs show __gen_oldest dominating CPU during ident drops.
      The stall caused connection pile-up, raising open session counts to 3-7k which further slowed the discard - a feedback loop.
      Single-ident discards held the schema and dhandle-list write locks for 16-70s.

      The discard path amplifies the cost but any generation check under high session counts pays the same linear scan.

      Possible directions (to be evaluated):

      • Cache/track the oldest generation instead of recomputing by full scan
      • Maintain a per-generation minimum incrementally as sessions enter/leave
      • Batch frees during file discard so one generation check covers many pages

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Yury Ershov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: