Stop persisting encoded ingest tombstones prefixed values on disk

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Layered Tables
    • None
    • Storage Engines - Foundations
    • 49.788
    • None
    • None

      Currently, the layered cursors implementation applies encoding logic for ingest tombstone-prefixed values (everything starting with \x14\x14) to both ingest and stable tables. This encoding logic is only required for ingest tables, but as a result, affected entries may be persisted on disk via the stable table, even though this is unnecessary.

      This is a problem because once data is persisted on disk, it becomes a data format compatibility guarantee and therefore cannot be removed. However, this logic is only needed for the current ingest tombstone design, and if we change that design in the future, it would no longer be required at all.

      Because of this, we should assess whether any customers have entries affected by this behavior and try to find a way to remove it as soon as possible.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Ivan Kochin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: