Selective Wander Join: Fast Progressive Visualizations for Data Joins

Abstract

Progressive visualization offers a great deal of promise for big data visualization; however, current progressive visualization systems do not allow for continuous interaction. What if users want to see more confident results on a subset of the visualization? This can happen when users are in exploratory analysis mode but want to ask some directed questions of the data as well. In a progressive visualization system, the online aggregation algorithm determines the database sampling rate and resulting convergence rate, not the user. In this paper, we extend a recent method in online aggregation, called Wander Join, that is optimized for queries that join tables, one of the most computationally expensive operations. This extension leverages importance sampling to enable user-driven sampling when data joins are in the query. We applied user interaction techniques that allow the user to view and adjust the convergence rate, providing more transparency and control over the online aggregation process. By leveraging importance sampling, our extension of Wander Join also allows for stratified sampling of groups when there is data distribution skew. We also improve the convergence rate of filtering queries, but with additional overhead costs not needed in the original Wander Join algorithm.

Big Data and Its Applications in Smart Real Estate and the Disaster Management Life Cycle: A Systematic Analysis

“…It is worth highlighting that data analyses are not limited to data visualizations only; however, the current paper focuses on visualization aspects of data analyses. Furthermore, as data continue growing bigger and bigger, traditional methods of information visualization are becoming outdated, inefficient, and handicapped in analyzing this enormously generated data, thus calling for global attention to develop better, more capable, and efficient methods for dealing with such big data [8,9]. Today, there is extensive use of real-time-based applications, whose procedures require real-time processing of the information for which advanced data visualization methods of learning are used.…”

Studying Early Decision Making with Progressive Bar Charts

“…An important aspect of visual data analysis is the interactive latency [33,50]. Progressive visualization techniques reduce interactive latency, thus improving user experience, by breaking down different stages of the visualization pipeline into iterative steps (e.g., [19,20,37,45,52]). These techniques have also been shown empirically to be effective at mitigating the latency in visual explorations [7,19,58].…”

“…In progressive visualizations, using animation to convey the uncertainty of intermediate results may conflict with the animation due to progressive updates. Despite the new designs developed for uncertainty visualizations, bar charts with error bars for confidence intervals remain the standard in many progressive visualization systems today [38,45,60]. We thus narrow the scope of our study to bar charts with error bars and their variations since they have not been evaluated yet in the progressive context and the proposed near-history representation also integrates more easily with the standard encoding than with e.g., violin plots.…”

“…Our third dataset was generated using the Wander Join algorithm [32] on the flights dataset [57] to get the flight delay data. Wander Join algorithm computes approximate results for join queries and has been the foundation of subsequent work on progressive aggregation over joins [45] owing to its suitability for OLAP use cases. The algorithm works by performing random walks.…”

“…Progressive bar charts are standard bar charts where the heights of the bars change at every update (e.g., every second) as estimates are refined until they settle to their final value. Many progressive visualization systems use bar charts [13,19,21,38,45,60], usually with confidence intervals shown as error bars to indicate estimate uncertainty. Prior work on uncertainty visualization has shown that uncertainty in data is difficult to understand in static visualizations [25,27], and in particular confidence intervals visualized as error bars are inaccurately interpreted for static bar charts [11].…”