I/O, performance analysis, and performance data immersion

Reed, Daniel A.; Madhyastha, Tara M.; Aydt, Ruth; Elford, C.L.; Scullin, Will H.; Smirni, Evgenia

doi:10.1109/mascot.1996.500987

Cited by 15 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Initial experiments suggest that this may be a useful method of visualizing an application's behaviour. Extensive 3D visualizations can be found in the Avatar system [ 20][ 21].…”

Section: Is To Have a Virtual Event Space That Contains Traces Ofmentioning

confidence: 99%

Experiences parallelizing, configuring, monitoring, and visualizing applications for clusters and multi-clusters

Anshus

Bjørndalen

Bongo

2004

Advances in Parallel Computing

View full text Add to dashboard Cite

To make it simpler to experiment with the impact different configurations can have on the performance of a parallel cluter application, we developed the PATHS system. The PATHS system use a "wrapper" to provide a level of indirection to the actual run-time location of data making the data available from wherever threads or processes are located. A wrapper specify where data is located, how to get there, and which protocols to use. Wrappers are also used to add or modify methods accessing data. Wrappers are specified dynamically. A "path" is comprised of one or more wrappers. Sections of a path can be shared among two or more paths.By reconfiguring the LAM-MPI Allreduce operation we achieved a performance gain of 1.52, 1.79, and 1.98 on respectively two, four and eight-way clusters. We also measured the performance of the unmodified Allreduce operation when using two clusters interconnected by a WAN link with 30-50ms roundtrip latency. Configurations which resulted in multiple messages being sent across the WAN did not add any significant performance penalty to the unmodified Allreduce operation for packet sizes up to 4KB. For larger packet sizes the Allreduce operation rapidly detoriated performancewise.To log and visualize the performance data we developed EventSpace, a configurable data collecting, management and observation system used for monitoring low-level synchronization and communication behavior of parallel applications on clusters and multi-clusters. Event collectors detect events, create virtual events by recording timestamped data about the events, and then store the virtual events to a virtual event space. Event scopes provide different views of the application, by combining and pre-processing the extracted virtual events. Online monitors are implemented as consumers using one or more event scopes. Event collectors, event scopes, and the virtual event space can be configured and mapped to the available resources to improve monitoring performance or reduce perturbation. Experiments demonstrate that a wind-tunnel application instrumented with event collectors, has insignificant slowdown due to data collection, and that monitors can reconfigure event scopes to trade-off between monitoring performance and perturbation. The visual views we generated allowed us to detect anomalous communication behavior, and detect load balance problems.

show abstract

“…Initial experiments suggest that this may be a useful method of visualizing an application's behaviour. Extensive 3D visualizations can be found in the Avatar system [ 20][ 21].…”

Section: Is To Have a Virtual Event Space That Contains Traces Ofmentioning

confidence: 99%

Experiences parallelizing, configuring, monitoring, and visualizing applications for clusters and multi-clusters

Anshus

Bjørndalen

Bongo

2004

Advances in Parallel Computing

View full text Add to dashboard Cite

show abstract

“…Figure 4(a) illustrates the dynamic behavior of the SAR code via a snapshot of the input phase. This snapshot is I taken from the Avatar performance data immersion system (Reed, Elford, Madhyastha, Scullin, et al, 1996;;, which supports real-time display of dynamic performance data and direct manipulation of application behavior.…”

Section: Syhithetlc Aperture Radar (Sar)mentioning

confidence: 99%

“…The efficacy of a particular choice depends on the relative contribution of these events to application behavior. The Pablo instrumentation library (Reed, Elford, Madhyastha, Scullin, et al, 1996) generates blocking (nonblocking) send begin events whenever a processor begins transmission of a blocking (nonblocking) message. For these two communication events, we chose the message length as the performance metric for clustering.…”

Section: Syhithetlc Aperture Radar (Sar)mentioning

confidence: 99%

“…To effectively reduce the total volume of performance data, statistical clustering must operate during program execution, capturing traces only from representative processors. Below, we briefly describe our experiences with an implementation of real-time statistical clustering that builds on the Pablo performance environment's instrumentation library (Reed, Elford, Madhyastha, Scullin, et al, 1996;. Figure 10 shows the integration of the Pablo performance instrumentation library with a real-time statistical clustering infrastructure.…”

Section: Real-time Clusteringmentioning

confidence: 99%

“…Each strikes a different balance among performance data volume, instrumentation perturbation, and I I I implementation complexity. By recording the detailed ¡ temporal and spatial patterns of software interactions, , timestamped event tracing subsumes the functions of pro-I filing, timing, and counting (i.e., one can generate pro-' files, counts, and timing summaries from event traces) (Reed, Elford, Madhyastha, Scullin, et al, 1996;Miller, Callaghan, et al, 1995;Reis et al, 1993). The flexibility of event tracing is not without price.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Statistical Clustering for Event Trace Reduction

Nickolayev

Roth²,

Reed

1997

The International Journal of Supercomputer Applications and Hig

View full text Add to dashboard Cite

Event tracing provides the detailed data needed to understand the dynamics of interactions among application resource demands and system responses. However, capturing the large volume of dynamic performance data inherent in detailed tracing can perturb program execution and stress secondary storage systems. Moreover, it can overwhelm a user or performance analyst with potentially irrelevant data. Using the Pablo performance environment's support for real-time data analysis, we show that dynamic statistical data clustering can dramatically reduce the volume of captured performance data by identifying and recording event traces only from representative processors. In turn, this makes possible low overhead, interactive visualization, and performance tuning.

show abstract