Accelerating Network Traffic Analytics Using Query-Driven Visualization

Bethel, E. Wes; Campbell, Scott; Dart, Eli; Stockinger, Kurt; Wu, Kesheng

doi:10.1109/vast.2006.261437

Cited by 27 publications

(27 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It implements the fastest known bitmap compression technique [37,38], and has been demonstrated to be effective in a number of data analysis applications [39,40]. In particular, it has a number of efficient functions for computing conditional histograms [41], which are crucial for this work.…”

Section: High Performance Index/query For Data Miningmentioning

confidence: 99%

See 1 more Smart Citation

Automatic beam path analysis of laser wakefield particle acceleration data

Rübel¹,

Geddes²,

Cormier‐Michel³

et al. 2009

Comput. Sci. Disc.

Self Cite

View full text Add to dashboard Cite

Abstract.Numerical simulations of laser wakefield particle accelerators play a key role in the understanding of the complex acceleration process and in the design of expensive experimental facilities. As the size and complexity of simulation output grows, an increasingly acute challenge is the practical need for computational techniques that aid in scientific knowledge discovery. To that end, we present a set of data-understanding algorithms that work in concert in a pipeline fashion to automatically locate and analyze high energy particle bunches undergoing acceleration in very large simulation datasets. These techniques work cooperatively by first identifying features of interest in individual timesteps, then integrating features across timesteps, and based on the information derived perform analysis of temporally dynamic features. This combination of techniques supports accurate detection of particle beams enabling a deeper level of scientific understanding of physical phenomena than has been possible before. By combining efficient data analysis algorithms and state-of-the-art data management we enable high-performance analysis of extremely large particle datasets in 3D. We demonstrate the usefulness of our methods for a variety of 2D and 3D datasets and discuss the performance of our analysis pipeline.

show abstract

Section: High Performance Index/query For Data Miningmentioning

confidence: 99%

“…The visualization is then focused on the selected features, thus significantly reducing the amount of data presented to the user. The concept of QDV has been successfully applied to a wide range of applications, e.g., network traffic analysis [39] and visual exploration of LWFA simulation data [7].…”

Section: Query-driven and Feature-based Visualizationmentioning

confidence: 99%

Automatic beam path analysis of laser wakefield particle acceleration data

Rübel¹,

Geddes²,

Cormier‐Michel³

et al. 2009

Comput. Sci. Disc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, query-driven visualization (QDV) integrates database technologies and visualization strategies to address the continually increasing size and complexity of scientific data [38][39][40]. In QDV, large data is intelligently pared down by user-specified selection queries, allowing smaller, more meaningful subsets of data to be efficiently analyzed and visualized.…”

Section: : End Ifmentioning

confidence: 99%

“…Accelerated processing performance times are critical for many scientific applications, e.g. query-driven visualization (QDV) [38][39][40], where data can be presumed to be read once, cached by the OS, and queried repeatedly during analysis stages. In these applications, user workloads are driven more by processing performance times, which make up a larger percentage of the analysis workload, then by disk access times.…”

Section: Answering a Simple Range Querymentioning

confidence: 99%

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Gosink

Bethel

et al. 2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community. The increase of cores at exponential rates is likely to affect virtually every server and client in the coming decade, and presents database management systems with a huge, compelling disruption that will radically change how processing is done. This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures. In our approach, our Data Parallel Bin-based Index Strategy (DP-BIS) first bins the base data, and then partitions and stores the values in each bin as a separate, bin-based data cluster. In answering a query, the procedures for examining the bin numbers and the bin-based data clusters offer the maximum possible level of concurrency; each record is evaluated by a single thread and all threads are processed simultaneously in parallel.We implement and demonstrate the effectiveness of DP-BIS on two multicore architectures: a multi-core CPU and a GPU. The concurrency afforded by DP-BIS allows us to fully utilize the thread-level parallelism provided by each architecture-for example, our GPU-based DP-BIS implementation simultaneously evaluates over 12,000 records with an equivalent number of concurrently executing threads. In comparing DP-BIS's performance across these architectures, we show that the GPU-based DP-BIS implementation requires significantly less computation time to answer a query than the CPU-based implementation. We also demonstrate in our analysis that DP-BIS provides better overall performance than the commonly utilized CPU and GPU-based projection index. Finally, due to data encoding, we show that DP-BIS accesses significantly smaller amounts of data than index strategies that operate solely on a column's base data; this smaller data footprint is critical for parallel processors that possess limited memory resources (e.g. GPUs).

show abstract

“…First, the notion of performing network traffic analysis using statistics (e.g., histograms) rather than raw data led to a methodology that enabled exploration and data mining at unprecedented speed [42]. That study showed the use of these concepts to rapidly detect a distributed scan attack on a dataset of unprecedented size -2.5 billion records.…”

Section: High-performance Query-driven Visualizationmentioning

confidence: 99%

Linking Automated Data Analysis and Visualization with Applications in Developmental Biology and High-Energy Physics

Ruebel

2009

View full text Add to dashboard Cite

Knowledge discovery from large and complex collections of today's scientific datasets is a challenging task. With the ability to measure and simulate more processes at increasingly finer spatial and temporal scales, the increasing number of data dimensions and data objects is presenting tremendous challenges for data analysis and effective data exploration methods and tools. Researchers are overwhelmed with data and standard tools are often insufficient to enable effective data analysis and knowledge discovery.The main objective of this thesis is to provide important new capabilities to accelerate scientific knowledge discovery form large, complex, and multivariate scientific data. The research covered in this thesis addresses these scientific challenges using a combination of scientific visualization, information visualization, automated data analysis, and other enabling technologies, such as efficient data management. The effectiveness of the proposed analysis methods is demonstrated via applications in two distinct scientific research fields, namely developmental biology and high-energy physics.Advances in microscopy, image analysis, and embryo registration enable for the first time measurement of gene expression at cellular resolution for entire organisms. Analysis of highdimensional spatial gene expression datasets is a challenging task. By integrating data clustering and visualization, analysis of complex, time-varying, spatial gene expression patterns and their formation becomes possible. The analysis framework MATLAB and the visualization have been integrated, making advanced analysis tools accessible to biologist and enabling bioinformatic researchers to directly integrate their analysis with the visualization.Laser wakefield particle accelerators (LWFAs) promise to be a new compact source of highenergy particles and radiation, with wide applications ranging from medicine to physics. To gain insight into the complex physical processes of particle acceleration, physicists model LWFAs computationally. The datasets produced by LWFA simulations are (i) extremely large, (ii) of varying spatial and temporal resolution, (iii) heterogeneous, and (iv) high-dimensional, making analysis and knowledge discovery from complex LWFA simulation data a challenging task. To address these challenges this thesis describes the integration of the visualization system VisIt and the state-of-the-art index/query system FastBit, enabling interactive visual exploration of extremely large three-dimensional particle datasets. Researchers are especially interested in beams of high-energy particles formed during the course of a simulation. This thesis describes novel methods for automatic detection and analysis of particle beams enabling a more accurate and efficient data analysis process. By integrating these automated analysis methods with visualization, this research enables more accurate, efficient, and effective analysis of LWFA simulation data than previously possible. Zusammenfassung

show abstract

Accelerating Network Traffic Analytics Using Query-Driven Visualization

Cited by 27 publications

References 29 publications

Automatic beam path analysis of laser wakefield particle acceleration data

Automatic beam path analysis of laser wakefield particle acceleration data

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Linking Automated Data Analysis and Visualization with Applications in Developmental Biology and High-Energy Physics

Contact Info

Product

Resources

About