Optimal distance bounds for fast search on compressed time-series query logs

Abstract

Consider a database of time-series, where each datapoint in the series records the total number of users who asked for a specific query at an internet search engine. Storage and analysis of such logs can be very beneficial for a search company from multiple perspectives. First, from a data organization perspective, because query Weblogs capture important trends and statistics, they can help enhance and optimize the search experience (keyword recommendation, discovery of news events). Second, Weblog data can provide an important polling mechanism for the microeconomic aspects of a search engine, since they can facilitate and promote the advertising facet of the search engine (understand what users request and when they request it).Due to the sheer amount of time-series Weblogs, manipulation of the logs in a compressed form is an impeding necessity for fast data processing and compact storage requirements. Here, we explicate how to compute the lower and upper distance bounds on the time-series logs when working directly on their compressed form. Optimal distance estimation means tighter bounds, leading to better candidate selection/elimination and ultimately faster search performance. Our derivation of the optimal distance bounds is based on the careful analysis of the problem using optimization principles. The experimental evaluation suggests a clear performance advantage of the proposed method, compared to previous compression/search techniques. The presented method results in a 10-30% improvement on distance estimations, which in turn leads to 25-80% improvement on the search performance. 6:2 • M. Vlachos et al. ACM Reference Format: Vlachos, M., Kozat, S. S., and Yu, P. S. 2010. Optimal distance bounds for fast search on compressed time-series query logs.

Compressive mining: fast and optimal data mining in the compressed domain

“…This is shown in Figure 2a). In [12,11], the authors advocated the use of high-energy coefficients and side information on the discarded coefficients for weblog sequence repositories; in that setting one of the sequences was compressed, whereas the query was uncompressed, i.e., all coefficients were available as illustrated in Figure 2b). This work examines the most general and challenging case when both objects are compressed.…”

tsdb: A Compressed Database for Time Series

“…The rationale of these methods is to represent the signal in the frequency domain instead of the time domain to capture important signal properties such as periodicity. By capturing predominant patterns, lossy time series compression techniques enable operations such as nearest neighbor searches, and pattern searches [5] directly in the compressed domain, that are not possible with lossless compression techniques unless additional indexes are used. The main drawback of lossy compression techniques is that they rely on specific patterns for providing a good approximation of the given time series.…”

Optimal Distance Estimation Between Compressed Data Series

“…This is shown in Figure 3a). In [11,12], the authors advocated the use of high-energy coefficients and side-information on the discarded coefficients for weblog sequence repositories; in that setting one of the sequences was compressed, whereas the query was uncompressed, i.e., all coefficients were available as illustrated in Figure 3b). This work examines the most general and challenging case when both series are compressed.…”

“…a) Both X,Q are compressed by storing the first coefficients. b) Using the highest-energy coefficients for X, whereas Q is uncompressed as in [11,12] , and c) the problem we address: both sequences are compressed using the highest-energy coefficients. Note that in general for each object a different set of coefficients is used.…”