Near-Lossless Compression for Large Traffic Networks

Abstract

Abstract-With advancements in sensor technologies, intelligent transportation systems (ITS) can collect traffic data with high spatial and temporal resolution. However, the size of the networks combined with the huge volume of the data puts serious constraints on the system resources. Low-dimensional models can help ease these constraints by providing compressed representations for the networks. In this study, we analyze the reconstruction efficiency of several low-dimensional models for large and diverse networks. The compression performed by low-dimensional models is lossy in nature. To address this issue, we propose a near-lossless compression method for traffic data by applying the principle of lossy plus residual coding. To this end, we first develop low-dimensional model of the network. We then apply Huffman coding in the residual layer. The resultant algorithm guarantees that the maximum reconstruction error will remain below a desired tolerance limit. For analysis, we consider a large and heterogeneous test network comprising of more than 18000 road segments. The results show that the proposed method can efficiently compress data obtained from a large and diverse road network, while maintaining the upper bound on the reconstruction error.

Highly Efficient Lossless Coding for High Dynamic Range Red, Clear, Clear, Clear Image Sensors

“…Currently, most of automotive applications, mainly due to the broadly understood safety, do not allow any lossy data compression (the processed data must be identical to the source data). Unfortunately, lossless compression techniques cannot fulfill the need of high compression ratio [ 7 , 8 ]. Even the compressed data (especially this losslessly compressed) is a huge problem both for onboard storage and for wireless transmission [ 9 ] (the high throughput of the 5G wireless transmission standard is surely needed in this case, however it is still in a preliminary stage in the automotive field [ 10 ]).…”

Sparse Low-Rank Matrix Approximation for Data Compression

“…At the first glance, applying LRMA to data compression seems to be straightforward, since one only needs to store k(m + n) elements, with small approximation error introduced in LRMA. Such an idea has been used extensively to compress various types of data, e.g., images/videos [2], [16], [17], [18], [19], [20], [3], 3D motion data [21], [22], [23], [24], [25], traffic data [26], [27], [28]. However, data samples usually exhibit both intra-coherence (i.e., coherence within each data sample) and inter-coherence (i.e., coherence among different data samples).…”

“…However, data samples usually exhibit both intra-coherence (i.e., coherence within each data sample) and inter-coherence (i.e., coherence among different data samples). LRMA can exploit the inter-coherence well, i.e., using B with much smaller orthogonal columns to represent X, but it fails to address the intra-coherence in the columns of B, and hereby compromises the compression performance [21], [22], [25], [26], [27], [28]. Figure 2(a) shows the problem using a 2D image set.…”

Low-dimensional models for compression, estimation and prediction of large-scale traffic data

“…Low-dimensional representation of large traffic data sets might be obtained using various techniques. In the recent study, M. T. Asif et al analyzed the reconstruction efficiency of singular value decomposition (SVD), discrete cosine transform (DCT), wavelet transform and non-negative matrix factorization for low-dimensional representation [1]. In addition, the authors proposed a near-lossless compression algorithm based on Huffman coding on low-dimensional network space.…”

“…Moreover, even if we obtain the basis vectors from historical data, we still need to collect data from all sensors during online operation. Therefore, PCA and the method studied in [1] are often applied for offline operations such as compression, future selection and missing data imputation [3,6,7,8,9].…”