ATLAS: A scalable and high-performance scheduling algorithm for multiple memory controllers

2020

In heterogeneous multicore systems, the memory subsystem plays a critical role, since most core-to-core communications are conducted through the main memory. Memory efficiency has a substantial impact on system performance. Although memory traffic from multimedia cores generally manifests high row-buffer locality, which is beneficial to memory efficiency, the locality is often lost as memory streams are forwarded through networks-on-chip (NoC). Previous studies have discussed the techniques that improve memory visibility to reveal scattered row-buffer hit opportunities to the memory scheduler. However, extending local memory visibility introduces little benefit after the locality has been severely diluted. As the alternative approach, preserving row-buffer locality in the NoC has not been well explored. What is worse, it remains to be studied how to perform network traffic scheduling with the awareness of both memory efficiency and quality-of-service (QoS). In this article, we propose a router design with embedded row-index caches to enable locality-aware packet forwarding. The proposed design requires minor modifications to existing router microarchitecture and can be easily implemented with priority arbiters to integrate QoS support. Extensive evaluations show that the proposed design achieves higher memory efficiency than prior memory-aware routers, in addition to providing QoS support. On basis of extant QoS-aware routers, locality-aware forwarding helps to increase row-buffer hits by 58.32% and reduce memory latency by 14.45% on average. It also introduces a net reduction in DRAM and NoC energy cost by 27.82%.

Section: Related Workmentioning

confidence: 99%

Improving Memory Efficiency in Heterogeneous MPSoCs through Row-Buffer Locality-aware Forwarding

Song

2020

“…PAR-BS [46] processes memory requests in batch to avoid unfairness and it exploits bank-level parallelism to achieve high throughput. ATLAS [27] prioritizes the threads that have attained least from the memory scheduler. TCM [28] groups the threads into latency-sensitive and bandwidth-intensive clusters and prioritizes latency-sensitive clusters.…”

Section: Related Work Gpu Concurrent Kernel Executionmentioning

confidence: 99%

Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution

Dai

Mantor

et al. 2019

Contemporary GPUs support multiple kernels to run concurrently on the same streaming multiprocessors (SMs). Recent studies have demonstrated that such concurrent kernel execution (CKE) improves both resource utilization and computational throughput. Most of the prior works focus on partitioning the GPU resources at the cooperative thread array (CTA) level or the warp scheduler level to improve CKE. However, significant performance slowdown and unfairness are observed when latency-sensitive kernels co-run with bandwidth-intensive ones. The reason is that bandwidth over-subscription from bandwidth-intensive kernels leads to much aggravated memory access latency, which is highly detrimental to latency-sensitive kernels. Even among bandwidth-intensive kernels, more intensive kernels may unfairly consume much higher bandwidth than less-intensive ones. In this article, we first make a case that such problems cannot be sufficiently solved by managing CTA combinations alone and reveal the fundamental reasons. Then, we propose a coordinated approach for CTA combination and bandwidth partitioning. Our approach dynamically detects co-running kernels as latency sensitive or bandwidth intensive. As both the DRAM bandwidth and L2-to-L1 Network-on-Chip (NoC) bandwidth can be the critical resource, our approach partitions both bandwidth resources coordinately along with selecting proper CTA combinations. The key objective is to allocate more CTA resources for latencysensitive kernels and more NoC/DRAM bandwidth resources to NoC-/DRAM-intensive kernels. We achieve it using a variation of dominant resource fairness (DRF). Compared with two state-of-the-art CKE optimization schemes, SMK [52] and WS [55], our approach improves the average harmonic speedup by 78% and 39%, respectively. Even compared to the best possible CTA combinations, which are obtained from an exhaustive search among all possible CTA combinations, our approach improves the harmonic speedup by up to 51% and 11% on average.

“…The First-Ready First Come First Serve (FR-FCFS) policy [19] was introduced to deliver high memory throughput by prioritizing requests that are going to open rows. Applicationaware scheduling for CPU-only systems began to draw more attention later [7,11,12]. ATLAS [11] prioritizes applications with low memory intensity to improve system throughput.…”

Section: Related Work Memory Schedulingmentioning

confidence: 99%

“…Applicationaware scheduling for CPU-only systems began to draw more attention later [7,11,12]. ATLAS [11] prioritizes applications with low memory intensity to improve system throughput. Thread cluster memory scheduling [12] dynamically clusters applications into low and high memory-intensity clusters and improves system performance as well as fairness.…”

Section: Related Work Memory Schedulingmentioning

confidence: 99%

A Self-aware Resource Management Framework for Heterogeneous Multicore SoCs with Diverse QoS Targets

Song

Alavoine

2019

In modern heterogeneous MPSoCs, the management of shared memory resources is crucial in delivering end-to-end QoS. Previous frameworks have either focused on singular QoS targets or the allocation of partitionable resources among CPU applications at relatively slow timescales. However, heterogeneous MPSoCs typically require instant response from the memory system where most resources cannot be partitioned. Moreover, the health of different cores in a heterogeneous MPSoC is often measured by diverse performance objectives. In this work, we propose the Self-Aware Resource Allocation framework for heterogeneous MP-SoCs. Priority-based adaptation allows cores to use different target performance and self-monitor their own intrinsic health. In response, the system allocates non-partitionable resources based on priorities. The proposed framework meets a diverse range of QoS demands from heterogeneous cores. Moreover, we present a runtime scheme to configure priority-based adaptation so that distinct sensitivities of heterogeneous QoS targets with respect to memory allocation can be accommodated. In addition, the priority of best-effort cores can also be regulated. This article is an extension of a conference paper: "SARA: Self-Aware Resource Allocation for Heterogeneous MPSoCs," published in DAC'18 (Song et al.). In particular, we extend our work in the following ways: -We add a new section to discuss the challenges of priority-based adaptation in the SARA framework. To deal with the challenges, we introduce a two-stage runtime configuration solution, including distributed self-configuration and global regulation, to accommodate best-effort cores and real-time cores that are particularly sensitive to memory allocation updates.-We expand the evaluation section to include more details on our simulation platform. We also present evaluation results on the runtime configuration scheme for the SARA framework.-In addition, a more comprehensive review of related work is provided in this paper to summarize prior efforts on memory scheduling and management.16:2 Y. Song et al. INTRODUCTIONModern heterogeneous MPSoCs [15,17] have been widely deployed in mobile devices thanks to their energy efficiency. These MPSoCs typically integrate a diverse collection of cores. Figure 1 depicts an example of a heterogeneous MPSoC. Besides general-purpose cores like the CPU for running applications, most heterogeneous cores are dedicated to certain functions, such as the GPU, the DSP, and the display. These cores have diverse notions of Quality-of-Service (QoS). For example, the GPU measures target real-time performance in terms of frame rate, the DSP demands the memory latency to remain below a certain limit, and the display requires sufficient bandwidth to refresh frames at a constant rate.To save cost and energy, heterogeneous cores commonly share resources, among which the sharing of the memory system including the network-on-chip (NoC) and the memory controller (MC) are the most challenging, because memory performance often has a direct and...