per-segment materialization concensus

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines - Server Integration
    • SESIRuby4Prezident 2026-08-25
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The PALI materialization concensus frontier is calculated globally, rather than per segment, which leads to the following kind of outage:

      • For segment 3: cells A and B are fully_materialized but C is not
      • For segment 5: A and C are fully_materialized, and B is not.

      Individually both segments 3 and 5 are "healthy" (2/3 completed), but globally PALI will perceive a mat outage where cell C is pinned in segment 3 and B is pinned at 5.

      In order to solve this problem we could take the concensus frontier for each segment individually (i.e. both 3 and 5 are concensus-fully-materialized so the global log should be).

      The immediate problem with the above is to do with page read routing. In thie above situation you want to route segment 3 reads around C and segment 5 reads around B but the PSR doesn't have access to segment information or LSN boundary information.

            Assignee:
            Unassigned
            Reporter:
            Nic Hollingum
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: