Gotcha: An Function-Wrapping Interface for HPC Tools

Poliakoff, David; Legendre, Matthew

doi:10.1007/978-3-030-17872-7_11

Cited by 4 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instrumentation profilers effectively inject additional instructions into the binary and are implemented through several methods: manually, automatic source-level (tool that modifies sourcecode), compiler-assisted, binary translation (tool that modifies compiled binary), runtime instrumentation (tool that supervises and controls execution after temporarily injecting instrumentation), and runtime injection (a lightweight form of runtime instrumentation that instruments jumps to helpers functions). At present, timemory supports manual instrumentation, runtime instrumentation for dynamically-linked binaries via Gotcha [23], binary translation, runtime instrumentation, and a simple command-line execution wrapper similar to the UNIX command-line tool time except with extensions to include memory and I/O values and rates and hardware counters. Additionally, timemory distributes a number of "instrumentation libraries" which provide simple function interfaces for activating instrumentation around performance monitoring APIs exposed by several commonly-used APIs, e.g., Kokkos, MPI, and OpenMP.…”

Section: Profiling Capabilitiesmentioning

confidence: 99%

Timemory: Modular Performance Analysis for HPC

Madsen

Awan

Brunie

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

HPC has undergone a significant transition toward heterogeneous architectures. This transition has introduced several issues in code migration to support multiple frameworks for targeting the various architectures. In order to cope with these challenges, projects such as Kokkos and LLVM create abstractions which map a generic frontend API to the backend that supports the targeted architecture. This paper presents a complementary framework for performance measurement and analysis. Several performance measurement and analysis tools in existence provide their capabilities through various methods but the common theme among these tools are prohibitive limitations in terms of user-level extensions. For this reason, software developers commonly have to learn multiple tools and valuable analysis methods, such as the roofline model, are frequently required to be generated manually. The timemory framework provides complete modularity for performance measurement and analysis and eliminates all restrictions on user-level extensions. The timemory framework also provides a highly-efficient and intuitive method for handling multiple tools/measurements (i.e., "components") concurrently. The intersection of these characteristics provide ample evidence that timemory can serve as the common interface for existing performance measurement and analysis tools. Timemory components are developed in C++ but includes multi-language support for C, Fortran, and Python codes. Numerous components are provided by the library itself -including, but not limited to, timers, memory usage, hardware counters, and FLOP and instruction roofline models. Additionally, analysis of the intrinsic overhead demonstrates superior performance in comparison with popular tools.

show abstract

Section: Profiling Capabilitiesmentioning

confidence: 99%

Timemory: Modular Performance Analysis for HPC

Madsen

Awan

Brunie

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Libsysio provides a POSIX-like interface that redirects I/O function calls to file systems and supports the conventional VFS/vnode architecture for file-based accesses[37]. Gotcha is a wrapper library that works similar to LD PRELOAD but operates via a programmable API[60]. † † https://github.com/pmem/syscall_intercept4.2 FUSE file systemsThe FUSE (Filesystem in Userspace)[64] [77] framework is another popular approach when developing user-space file systems.…”

mentioning

confidence: 99%

Ad Hoc File Systems for High-Performance Computing

Brinkmann

Mohror

et al. 2020

J. Comput. Sci. Technol.

View full text Add to dashboard Cite

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include node-local SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 "Challenges and Opportunities of User-Level File Systems for HPC" and discusses application scenarios as well as design strategies for ad hoc file systems using nodelocal storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

show abstract