Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-57438

XOR based compression for doubles

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 5.1 Required
    • Component/s: None
    • Labels:
      None
    • Sprint:
      Execution Team 2021-06-14

      Description

      Implement an encoder and decoder for the XOR based compression for double-precision floating-points as described in the "Gorilla: A Fast, Scalable, In-Memory Time Series Database" paper by Pelkonen et al.

      Doubles are XOR'ed with previous value and number of leading and trailing zeroes in the result are counted. The remaining "value" bits are called meaningful bits.

      A bit stream is encoded with the following control bits followed by data:

      '0': XOR yieleded zero (value is same as previous), nothing more to store.

      '10': If number of leading and trailing zeros are at least as many as previous '11' state, we reuse these bits and store just the meaningful bits.

      '11': count of leading zeros are stored in the next 5 bits, then number of meaningful bits are stored in the next 6 bits and last the actual meaningful bits are stored.

        Attachments

          Activity

            People

            Assignee:
            backlog-server-execution Backlog - Storage Execution Team
            Reporter:
            henrik.edin Henrik Edin
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated: