Staged memory scheduling: Achieving high performance and scalability in heterogeneous systems

Abstract

When multiple processor (CPU) cores and a GPU integrated together on the same chip share the off-chip main memory , requests from the GPU can heavily interfere with requests from the CPU cores, leading to low system performance and starvation of CPU cores. Unfortunately, state-of-the-art application-aware memory scheduling algorithms are ineffective at solving this problem at low complexity due to the large amount of GPU traffic. A large and costly request buffer is needed to provide these algorithms with enough visibility across the global request stream, requiring relatively complex hardware implementations. This paper proposes a fundamentally new approach that de-couples the memory controller's three primary tasks into three significantly simpler structures that together improve system performance and fairness, especially in integrated CPU-GPU systems. Our three-stage memory controller first groups requests based on row-buffer locality. This grouping allows the second stage to focus only on inter-application request scheduling. These two stages enforce high-level policies regarding performance and fairness, and therefore the last stage consists of simple per-bank FIFO queues (no further command reordering within each bank) and straightforward logic that deals only with low-level DRAM commands and timing. We evaluate the design trade-offs involved in our Staged Memory Scheduler (SMS) and compare it against three state-of-the-art memory controller designs. Our evaluations show that SMS improves CPU performance without degrading GPU frame rate beyond a generally acceptable level, while being significantly less complex to implement than previous application-aware schedulers. Furthermore, SMS can be configured by the system software to prioritize the CPU or the GPU at varying levels to address different performance needs.

Bounding memory interference delay in COTS-based multi-core systems

“…In case of RD-to-WR, WR needs to wait CL + BL/2 + 2 − WL cycles. 6 Therefore, for a WR/RD command, the maximum delay from each of the commands that have arrived earlier is:…”

“…If one of the tasks accesses a row different from the currently open one, this memory access causes a row-conflict request so that the re-ordering effect no longer occurs. In many systems, as described in [27,25,6], the re-ordering effect can also be bounded by a hardware threshold N cap , which caps the number of re-ordering between requests. Therefore, the maximum number of row-hits that can be prioritized over older row-conflicts is:…”

DRAMSys: A Flexible DRAM Subsystem Design Space Exploration Framework

“…The scheduler module collects transactions and reorders them with respect to latency and power savings and issues them to the backend with the channel controller that takes care of the correct use of the DRAM. DRAMSys supports state-of-the-art scheduling algorithms, FR-FCFS [21], Par-BS [22] and SMS [23] or it can simply disable the scheduling unit. Furthermore, the model has a Reorder Buffer (ROB) to provide in-order responses to the requester and it also c 2015 Information Processing Society of Japan supports a multi-rank configuration of the DRAM subsystem.…”

“…This refresh aware policy can also be applied to more recent DRAM schedulers like Refs. [22] and [23] and can also be used c 2015 Information Processing Society of Japan …”

Adaptive Memory Controller for High-performance Multi-channel Memory

“…In order to mitigate this issue, previous studies have explored different approaches including thread scheduling, memory channel partitioning, and address mapping schemes. Thread scheduling techniques are studied in various texts [1,[3][4][5][6]. Thread scheduling approaches prioritize memory access requests from different applications (or threads) to give more priority to those whose performance gain could be maximized.…”