Space-efficient data structures for Top- k completion

Abstract

Virtually every modern search application, either desktop, web, or mobile, features some kind of query auto-completion. In its basic form, the problem consists in retrieving from a string set a small number of completions, i.e. strings beginning with a given prefix, that have the highest scores according to some static ranking. In this paper, we focus on the case where the string set is so large that compression is needed to fit the data structure in memory. This is a compelling case for web search engines and social networks, where it is necessary to index hundreds of millions of distinct queries to guarantee a reasonable coverage; and for mobile devices, where the amount of memory is limited.We present three different trie-based data structures to address this problem, each one with different space/time/ complexity trade-offs. Experiments on large-scale datasets show that it is possible to compress the string sets, including the scores, down to spaces competitive with the gzip'ed data, while supporting efficient retrieval of completions at about a microsecond per completion.

Design of Practical Succinct Data Structures for Large Data Collections

“…On large sets of queries and URLs, experiments have shown compression ratios close or better than those of gzip, with average times per completion of about one microsecond [16].…”

A two-dimensional click model for query auto-completion

“…For example, [4,10] studied the space efficiency of index for QAC. [21,11] investigated the efficient algorithms for QAC.…”

“…We observe very similar phenomenon in the Iphone 5 platform. [1,5] 39,405 57.6% Sess with FPL in [6,10] 39,882 64.8% Sess with FPL in [11,15] 22,892 59.1% Sess with FPL in [16,50] 23,213 60.2%…”

“…With the prevalence of mobile devices, it becomes more critical because typing takes more effort in mobile devices than in PCs. Previous studies addressed the QAC problem in different perspectives, ranging from designing more efficient indexes and algorithms [4,10,21,11], leveraging context in long and short term query history [3], learning to combine more personalized signals such as gender, age and location [19], suggesting queries from a mis-spelled prefix [7].…”

Encodings for Range Selection and Top-k Queries

“…An example is auto-completion search in databases and search engines [13,15]. Here, as the user types in a query, the system presents the user with the k most popular completions, chosen from a lexicon of phrases, based on the text entered so far.…”