CAMEO: Autocorrelation-Preserving Line Simplification for Lossy Time Series Compression
Time series data from a variety of sensors and IoT devices need effective compression to reduce storage and I/O bandwidth requirements. While most timeseries databases and systems rely on lossless compression, lossy techniques offer even greater space saving with a small loss in precision. However, the unknown impact on downstream analytics requires a semi-manual trial-and-error exploration. We initiate work on lossy compression that provides guarantees on complex statistical features (which are strongly correlated with the accuracy of downstream analytics). Specifically, we propose CAMEO, a new lossy compression method that provides guarantees on the autocorrelation and partial-autocorrelation functions (ACF/PACF) of a time series. Our method leverages line simplification techniques as well as incremental maintenance, blocking, and parallelization strategies for effective and efficient compression. The results show that our method improves compression ratios by 2x on average and up to 54x on selected datasets, compared to previous lossy and lossless compression methods. Moreover, we maintain, and sometimes even improve, the forecasting accuracy by preserving the autocorrelation properties of the time series.
