Dissecting On-Node Memory Access Performance: A Semantic Approach

Abstract

Abstract-Optimizing memory access is critical for performance and power efficiency. CPU manufacturers have developed sampling-based performance measurement units (PMUs) that report precise costs of memory accesses at specific addresses. However, this data is too low-level to be meaningfully interpreted and contains an excessive amount of irrelevant or uninteresting information.We have developed a method to gather fine-grained memory access performance data for specific data objects and regions of code with low overhead and attribute semantic information to the sampled memory accesses. This information provides the context necessary to more effectively interpret the data. We have developed a tool that performs this sampling and attribution and used the tool to discover and diagnose performance problems in real-world applications. Our techniques provide useful insight into the memory behavior of applications and allow programmers to understand the performance ramifications of key design decisions: domain decomposition, multi-threading, and data motion within distributed memory systems.

A Topology-Aware Performance Monitoring Tool for Shared Resource Management in Multicore Systems

“…5 Analyzing tasks concurrency gives a room for thread placement Here, we will illustrate a situation where a visual hint can be useful to choose thread placement. In this example the monitoring mode is the per node one: we monitor all the processes of the node.…”

“…Former approaches were often less portable or do not expose as many details about cache sharing etc. MemAxes [5] oers ne-grained memory performance analysis with a graphical radial hierarchy display. However, it only focuses on static post-mortem analysis of memory accesses while our approach is dynamic and works for all performance metrics and more kinds of resource sharing.…”

Automating the Application Data Placement in Hybrid Memory Systems

“…7 As reported by each process by the Virtual Memory High-Water Mark (VmHWM) in /proc/self/status before the process termination. 8 The overhead is calculated using the reported FOM.…”

“…Precise-Event Based Sampling (PEBS) is the implementation of such a feature in recent Intel processors [6]. Performance analysis tools such as HPCToolkit [7], MemAxes [8], Extrae [9], and Intel R Vtune TM Amplifier [10] use a hybrid approach combining instrumentation to track data allocation and PEBS to monitor the application data references. This approach enables exploring in-production executions with a reduced overhead at the cost of providing statistical approximations, even though approximations for long runs resemble the actual results.…”

A prototype sampling interface for PAPI

“…The MemAxes tool described in [2] provides visualizations of NUMA data in semantic contexts, including node hardware topology, source code, and application-specific. These visualizations assist the programmer in finding and fixing NUMA performance bottlenecks.The visualization part of the MemAxes tool has been released for general use, but not the measurement part, because the authors have written their own kernel module to access the low-level NUMA performance data.…”