Adaptive processing of spatial-keyword data over a distributed streaming cluster

Abstract

The widespread use of GPS-enabled smartphones along with the popularity of micro-blogging and social networking applications, e.g., Twitter and Facebook, has resulted in the generation of huge streams of geo-tagged textual data. Many applications require real-time processing of these streams. For example, location-based e-coupon and ad-targeting systems enable advertisers to register millions of ads to millions of users. The number of users is typically very high and they are continuously moving, and the ads change frequently as well. Hence sending the right ad to the matching users is very challenging. Existing streaming systems are either centralized or are not spatial-keyword aware, and cannot efficiently support the processing of rapidly arriving spatial-keyword data streams. This paper presents Tornado, a distributed spatial-keyword stream processing system. Tornado features routing units to fairly distribute the workload, and furthermore, co-locate the data objects and the corresponding queries at the same processing units. The routing units use the Augmented-Grid, a novel structure that is equipped with an efficient search algorithm for distributing the data objects and queries. Tornado uses evaluators to process the data objects against the queries. The routing units minimize the redundant communication by not sending data updates for processing when these updates do not match any query. By applying dynamically evaluated cost formulae that continuously represent the processing overhead at each evaluator, Tornado is adaptive to changes in the workload. Extensive experimental evaluation using spatio-textual range queries over real Twitter data indicates that Tornado outperforms the nonspatio-textually aware approaches by up to two orders of magnitude in terms of the overall system throughput.

Optimizing Continuous kNN Queries over Large-Scale Spatial-Textual Data Streams

“…These indexes map the spatial search range to a hash bucket or grid. Several works ( [10][11][12]) study evaluating CQST on distributed server clusters.…”

“…As a generic continuous query over spatial-textual data streams (CQST), CkQST can be evaluated with the evaluation framework of CQST, i.e. according to the features of queries, selecting an appropriate spatial index and a textual index, and exploiting them with some appropriate filtering strategies to process the incoming objects [1][2][3][4][5][6][7][8][9][10][11][12].…”

Continuous k Nearest Neighbor Queries over Large-Scale Spatial–Textual Data Streams

“…A stream of spatial-textual objects (e.g., e-coupons or Weibos) generated are fed to the relevant users. Continuous queries over spatial-textual data streams studied by existing work [1][2][3][4][5][6][7][8][9][10][11][12] are primarily in terms of Boolean matching or approximate matching, which return an unpredictable number of objects or approximate results. The number of qualified objects containing all keywords specified by a user can be far larger than k, because the objects (e.g., tweets, news) usually contain much more keywords than queries do.…”

“…For 1 q , the spatial search range, thereafter "search range", is defined as a minimal circle centered at the geo- Challenges. The solution framework for evaluating generic continuous queries over spatialtextual data streams consists of selecting an appropriate spatial index and a textual index to form a hybrid spatial-textual index, and exploiting it with appropriate spatial and/or textual filtering strategies to process the incoming objects according to the features of queries [1][2][3][4][5][6][7][8][9][10][11][12]. There are three key challenges in constructing such an index for CkQST.…”

“…(1) Queries are mapped to the leaf nodes in the spatial index, which minimizes the spatial region of the nodes covering the search range of queries to reduce the number of objects to be verified [2,5,[7][8][9][10][13][14][15]. (2) Queries are mapped to the spatial nodes according to the spatial distribution [10][11][12], the keyword distribution [6], or the corresponding cost model [1,3,4]. These approaches are appropriate in the scenarios where the search range of the queries rarely changes, but inappropriate to CkQST, where frequent update of the search range of the queries results in high costs.…”