Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-9182

Explore what should be the correct way to calculate upd_memsize in the durable history era

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • 8
    • StorEng - Defined Pipeline

      Summary
      upd_memsize is calculated as the total size of update chain saved in reconciliation and is used as a heuristic to decide whether we need to split the page or not. Before durable history, it only contains the size of update chain that needs to be restored. While in durable history era, this value becomes larger because it also includes the size of the update chains that are moved to the history store.

      I tried to made a change to only count the size of uncommitted updates and that leads to more cache pressure compared to before the change.

      We need to explore what is the best way to calculate the update chain size to fine tune the heuristic we have.

      Motivation

      • Does this affect any team outside of WT?
        No
      • How likely is it that this use case or problem will occur?
        N/A
      • If the problem does occur, what are the consequences and how severe are they?
        With the current implementation, I think we are splitting more than before durable history. We need to understand its performance impact.
      • Is this issue urgent?
        No

      Acceptance Criteria (Definition of Done)
      Testing the performance of the system with different ways of calculating upd_memsize and decide which is the best strategy.

      • Testing
        As above
      • Documentation update
        Update the comment is enough.

      [Optional] Suggested Solution
      I have tried only calculating the size of uncommitted updates. The other way we can do is to only calculate the size of update chains that are going to be restored.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            chenhao.qu@mongodb.com Chenhao Qu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: