-
Type: Task
-
Resolution: Won't Do
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Storage Execution
-
Execution Team 2021-06-14
Implement an encoder and decoder for the XOR based compression for double-precision floating-points as described in the "Gorilla: A Fast, Scalable, In-Memory Time Series Database" paper by Pelkonen et al.
Doubles are XOR'ed with previous value and number of leading and trailing zeroes in the result are counted. The remaining "value" bits are called meaningful bits.
A bit stream is encoded with the following control bits followed by data:
'0': XOR yieleded zero (value is same as previous), nothing more to store.
'10': If number of leading and trailing zeros are at least as many as previous '11' state, we reuse these bits and store just the meaningful bits.
'11': count of leading zeros are stored in the next 5 bits, then number of meaningful bits are stored in the next 6 bits and last the actual meaningful bits are stored.