Heterogeneous-race-free memory models

Abstract

Commodity heterogeneous systems (e.g., integrated CPUs and GPUs), now support a unified, shared memory address space for all components. Because the latency of global communication in a heterogeneous system can be prohibitively high, heterogeneous systems (unlike homogeneous CPU systems) provide synchronization mechanisms that only guarantee ordering among a subset of threads, which we call a scope. Unfortunately, the consequences and semantics of these scoped operations are not yet well understood. Without a formal and approachable model to reason about the behavior of these operations, we risk an array of portability and performance issues.In this paper, we embrace scoped synchronization with a new class of memory consistency models that add scoped synchronization to data-race-free models like those of C++ and Java. Called sequential consistency for heterogeneousrace-free (SC for HRF), the new models guarantee SC for programs with "sufficient" synchronization (no data races) of "sufficient" scope. We discuss two such models. The first, HRF-direct, works well for programs with highly regular parallelism. The second, HRF-indirect, builds on HRFdirect by allowing synchronization using different scopes in some cases involving transitive communication. We quantitatively show that HRF-indirect encourages forward-looking programs with irregular parallelism by showing up to a 10% performance increase in a task runtime for GPUs.

Synchronization Using Remote-Scope Promotion

“…First, we describe how acquire/release synchronization works in C++. Next, we describe how HSA extends these semantics with scopes [10] [11]. It is common to couple a synchronization operation with a memory instruction (e.g., load acquire, store release, etc.).…”

“…Sequentially consistent for heterogeneous race free (SC for HRF) memory models were proposed to help programmers reason about scoped synchronization [10]. They extend sequentially consistent for data race free (SC for DRF) memory models with scoping rules.…”

“…When using scoped synchronization, programmers must manage both the level of visibility and the order of memory operations [10]. This is different than non-scoped synchronization, which only requires programmers to reason about order.…”

“…Because we leverage hierarchical memory that provides transitivity, a simple extension of HRF-indirect suffices [10]. Specifically, we build on an extended version of HRF-indirect called HRF-indirect-relaxed [11].…”

“…For instance, Hower et al [10] previously identified that race-free code using scoped synchronization must observe a total order of RMWs (referred to as atomics in that work). More recently, Gaster et al [11] and the HSA memory model [3] formally defined this requirement by stating that a GPU implementation must provide a "coherent order" for all accesses to a single memory location.…”