LIKWID Monitoring Stack: A Flexible Framework Enabling Job Specific Performance monitoring for the masses

Abstract

Abstract-System monitoring is an established tool to measure the utilization and health of HPC systems. Usually system monitoring infrastructures make no connection to job information and do not utilize hardware performance monitoring (HPM) data. To increase the efficient use of HPC systems automatic and continuous performance monitoring of jobs is an essential component. It can help to identify pathological cases, provides instant performance feedback to the users, offers initial data to judge on the optimization potential of applications and helps to build a statistical foundation about application specific system usage. The LIKWID monitoring stack is a modular framework build on top of the LIKWID tools library. It aims on enabling job specific performance monitoring using HPM data, system metrics and applicationlevel data for small to medium sized commodity clusters. Moreover, it is designed to integrate in existing monitoring infrastructures to speed up the change from pure system monitoring to job-aware monitoring.

Using analog computers in today's largest computational challenges

“…We use the performance measure codes likwid (Hager et al, 2010;Röhl et al, 2017;Gruber et al, 2020) and perf (de Melo, 2010) to measure the overall floating-point operations (FLOP) and energy usage of the digital processor. For the Intel mobile processor, this provides a power consumption of P D = 10 W during computing.…”

TEEMon

“…Ball et al [27] first proposed a set of optimal algorithms for program profiling and instruction tracing to reduce the overhead of profilers. Hardware specific tools such as LIKWID [58,64] for x86 environments and MemProf [46] for NUMA systems make use of hardware counters allowing developers to explore optimization opportunities specific to the underlying system architecture. Linux perf [34] provides low-level system metrics by attaching to tracepoints, performance counters or probes similarly to eBPF [36].…”

TEEMon: A continuous performance monitoring framework for TEEs

“…Ball et al [27] first proposed a set of optimal algorithms for program profiling and instruction tracing to reduce the overhead of profilers. Hardware specific tools such as LIKWID [63,69] for x86 environments and MemProf [48] for NUMA systems make use of hardware counters allowing developers to explore optimization opportunities specific to the underlying system architecture. Linux perf [34] provides low-level system metrics by attaching to tracepoints, performance counters or probes similarly to eBPF [36].…”