Search algorithms for multiway spatial joins

Papadias, Dimitris; Arkoumanis, Dinos

doi:10.1080/13658810210138733

Cited by 3 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many cases, this approach results in poor performance. Papadias et al proposed several methods to approach multiway spatial joins [11,16,[18][19][20]33]. Their studies show that multiway spatial joins can be processed best by combining synchronous traversal with a pairwise method such as a plane-sweep or an index nested-loop algorithm [32].…”

Section: Background and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Zhao

et al. 2017

IJGI

View full text Add to dashboard Cite

Multiway spatial join plays an important role in GIS (Geographic Information Systems) and their applications. With the increase in spatial data volumes, the performance of multiway spatial join has encountered a computation bottleneck in the context of big data. Parallel or distributed computing platforms, such as MapReduce and Spark, are promising for resolving the intensive computing issue. Previous approaches have focused on developing single-threaded join algorithms as an optimizing and partition strategy for parallel computing. In this paper, we present an effective high-performance multiway spatial join algorithm with Spark (MSJS) to overcome the multiway spatial join bottleneck. MSJS handles the problem through cascaded pairwise join. Using the power of Spark, the formerly inefficient cascaded pairwise spatial join is transformed into a high-performance approach. Experiments using massive real-world data sets prove that MSJS outperforms existing parallel approaches of multiway spatial join that have been described in the literature.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

“…Multiway spatial join is an extension of spatial join with multiple-inputs [11]. For example, the spatial query of "finding all the forests crossed by a river in each state" is a typical example of a multiway spatial join, which analyzes spatial relations among "forests", "rivers" and "states" using 1.…”

Section: Introductionmentioning

confidence: 99%

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Zhao

et al. 2017

IJGI

View full text Add to dashboard Cite

show abstract

Generalized communication cost efficient multi-way spatial join: revisiting the curse of the last reducer

et al. 2020

View full text Add to dashboard Cite

Scheduling distributed multiway spatial join queries: optimization models and algorithms

Oliveira

Costa

Foulds

et al. 2023

International Journal of Geographical Information Science

View full text Add to dashboard Cite

As discussed in the paper, when defining a schedule to process a query we want to control the scheduling behavior concerning the usage of computational resources, i.e., processing and network capacity. Our evaluation showed that attempting to minimize both processing and network utilization creates conflicting objectives, in the sense that to achieve a better balance in the query execution (and consequently a reduced makespan), an extra cost is incurred to transfer partitions to machines that are underloaded. To address this issue, we introduced a parameter in the model, termed f , to specify the degree of preference for a more balanced execution or for a lower usage of network resources.From a practical perspective, if the only concern for query execution is the communication cost, selecting a value for f is trivial (f =0). Analogously, if the only concern is the makespan, we can readily select a sufficiently large value for f so as to ignore the communication cost in the objective function. Often, however, the computational environment imposes constraints in both makespan and communication costs, and selecting a value for f that achieves a balance between these two objectives results in a reasonable saving of computational resources. Each numerical instance of the SM model has a particular given value for f but determining a suitable value of f is a non-trivial task, as it depends on the values of the processing and communication costs (w j and c ij ).We address in this supplementary material the question of determining an appropriate value for f by studying the effects of variations on this value on the optimal schedule for SM. We demonstrate how to determine intervals of f values for which an optimal schedule for SM remains unchanged. By establishing these intervals, we can identify the shape of the objective function (1.1) and provide answers to some practical questions that arise when scheduling queries using the SM model, such as: i) is it possible to reduce the makespan of a query even further?, ii) if yes, what is the minimum makespan and the additional communication cost incurred?, iii) analogously, what is the increase in makespan if we have to reduce the communication cost on a low bandwidth network?, and iv) what is the maximum value for f for which a change in makespan occurs?For linear programming, a fully-developed theory exists to determine valid intervals for the parameters of a model, for which an optimal schedule remains unchanged. It is known as postoptimality analysis (Bertsimas and Tsitsiklis 1997). This theory, however, is not valid for integer linear programming, and it turned out that a similar theory for the integer case is inherently more challenging and yet not fully developed (Blair 1997). Notwithstanding, by adapting the CONTACT Thiago Borges de Oliveira.

show abstract

Search algorithms for multiway spatial joins

Cited by 3 publications

References 24 publications

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Generalized communication cost efficient multi-way spatial join: revisiting the curse of the last reducer

Scheduling distributed multiway spatial join queries: optimization models and algorithms

Contact Info

Product

Resources

About