Scalable Communication Trace Compression

Krishnamoorthy, Sriram; Agarwal, Khushbu

doi:10.1109/ccgrid.2010.111

Cited by 11 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reducing the trace size will decrease both the time overhead and memory usage, hence improving the scalability of our tool. We plan to investigate trace compression methods similar to the ones in Noeth et al [2006] and Krishnamoorthy and Agarwal [2010] for DR-MCAPI traces.…”

Section: Conclusion and Future Research Directionsmentioning

confidence: 99%

Deterministic replay for message-passing-based concurrent programs

Elwakil

Yang

2012

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

The Multicore Communications API (MCAPI) is a new message-passing API that was released by the Multicore Association. MCAPI provides an interface designed for closely distributed embedded systems with multiple cores on a chip and/or chips on a board. Similar to parallel programs in other domains, debugging MCAPI programs is a challenging task due to their nondeterministic behavior. In this article we present a tool that is capable of deterministically replaying MCAPI program executions, which provides valuable insight for MCAPI developers in case of failure.

show abstract

Section: Conclusion and Future Research Directionsmentioning

confidence: 99%

Deterministic replay for message-passing-based concurrent programs

Elwakil

Yang

2012

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

show abstract

“…Becker et al [49] use event traces to analyze the performance for large-scale programs. Though many works for trace compression are proposed [50], [51], [52], [53], tracing still often brings very large overhead which makes it nonsuitable for production environments. Profiling Profiling can extract the program's statistical information with very low overhead.…”

Section: 03% Tot_cyc Variance Reductionmentioning

confidence: 99%

ScalAna: Automating Scaling Loss Detection with Graph Analysis

Jin

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

Scaling a parallel program to modern supercomputers is challenging due to inter-process communication, Amdahl's law, and resource contention. Performance analysis tools for finding such scaling bottlenecks either base on profiling or tracing. Profiling incurs low overheads but does not capture detailed dependencies needed for root-cause analysis. Tracing collects all information at prohibitive overheads.In this work, we design SCALANA that uses static analysis techniques to achieve the best of both worlds -it enables the analyzability of traces at a cost similar to profiling. SCALANA first leverages static compiler techniques to build a Program Structure Graph, which records the main computation and communication patterns as well as the program's control structures. At runtime, we adopt lightweight techniques to collect performance data according to the graph structure and generate a Program Performance Graph. With this graph, we propose a novel approach, called backtracking root cause detection, which can automatically and efficiently detect the root cause of scaling loss. We evaluate SCALANA with real applications. Results show that our approach can effectively locate the root cause of scaling loss for real applications and incurs 1.73% overhead on average for up to 2,048 processes. We achieve up to 11.11% performance improvement by fixing the root causes detected by SCALANA on 2,048 processes.

show abstract

“…Larus proposes whole program paths (WPP) to capture a program's dynamic control flow, where an enhanced SEQUITUR algorithm is designed to compress acyclic path traces [19]. Krishnamoorthy et al present a trace compression algorithm that is largely based on SEQUITUR [18]. To fully exploit the pattern detection capability of SEQUITUR, this work performs trace compression at argument level instead of event level.…”

Section: Related Workmentioning

confidence: 99%

Elastic and scalable tracing and accurate replay of non-deterministic events

Mueller

2013

Proceedings of the 27th International ACM Conference on International Conference on Supercomputing

View full text Add to dashboard Cite

SCALATRACE represents the state-of-the-art of parallel application tracing for high performance computing (HPC). This paper presents SCALATRACE II, a next generation tracer that delivers even higher trace compression capability, even when events are not always regular. In this work, we contribute a spectrum of novel compression and replay techniques that are fundamentally different from our past approaches. SCALATRACE II features a redesigned low-level encoding scheme of trace data such that data elements are elastic and self-explanatory. With this new encoding scheme, trace compression is enhanced by introducing innovative intra-node and inter-node trace compression algorithms that guarantee high compression rates in a loop structure agnostic fashion. In practice, the improved compression scheme is particularly efficient for scientific codes that demonstrate inconsistent behavior across time steps and nodes. A novel approach is further contributed to probabilistically replay sequences of non-deterministic events. To assess the compression efficacy of SCALATRACE II, we conduct experiments not only with computational kernels but also a real-world application, the Parallel Ocean Program (POP). Compared to the first generation SCALATRACE, we observe key improvements on trace compression for benchmarks with inconsistent time step behavior and diverging task level behavior while retaining timing accuracy even under probabilistic replay.

show abstract

Scalable Communication Trace Compression

Cited by 11 publications

References 32 publications

Deterministic replay for message-passing-based concurrent programs

Deterministic replay for message-passing-based concurrent programs

ScalAna: Automating Scaling Loss Detection with Graph Analysis

Elastic and scalable tracing and accurate replay of non-deterministic events

Contact Info

Product

Resources

About