A memory scheduling strategy for eliminating memory access interference in heterogeneous system

Abstract

Multiple CPUs and GPUs are integrated on the same chip to share memory, and access requests between cores are interfering with each other. Memory requests from the GPU seriously interfere with the CPU memory access performance. Requests between multiple CPUs are intertwined when accessing memory, and its performance is greatly affected. The difference in access latency between GPU cores increases the average latency of memory accesses. In order to solve the problems encountered in the shared memory of heterogeneous multi-core systems, we propose a step-by-step memory scheduling strategy, which improve the system performance. The step-by-step memory scheduling strategy first creates a new memory request queue based on the request source and isolates the CPU requests from the GPU requests when the memory controller receives the memory request, thereby preventing the GPU request from interfering with the CPU request. Then, for the CPU request queue, a dynamic bank partitioning strategy is implemented, which dynamically maps it to different bank sets according to different memory characteristics of the application, and eliminates memory request interference of multiple CPU applications without affecting bank-level parallelism. Finally, for the GPU request queue, the criticality is introduced to measure the difference of the memory access latency between the cores. Based on the first ready-first come first served strategy, we implemented criticality-aware memory scheduling to balance the locality and criticality of application access.

Towards Computational Awareness in Autonomous Robots: An Empirical Study of Computational Kernels

“…Most of the GPU resources are underutilized while occupying a huge amount of shared memory space. This usage of a huge amount of memory calls for a conflict resolution in CPU and GPU memory access, which is investigated in very recent literature [32], [33], [34], and can be put into action using the data from this paper. In the Zotac system, which has a discrete GPU unlike the Xaviers, the CV kernels show much better performance and lower CPU bottleneck issue, however the Zotac platform is not viable for aerial robots.…”

“…Autonomous robots continue to suffer from strict limits on computation and power, especially for small aerial vehicles. From a resource management perspective, we therefore focus on profiling that can inform two primary techniques: (1) thread-level scheduling that prioritizes among multiple computational kernels competing for shared computational resources such as CPU, GPU, and memory [31,32,33,34]; and (2) the Dynamic Voltage and Frequency Scaling (DVFS) mechanism on modern processors that allows for finegrained frequency scaling and power management [35]. We aim to analyze each computational kernel for sustainability and predictability by showing their worst case behavior, average behavior, and the variation from them.…”

A Survey of Techniques for Reducing Interference in Real-Time Applications on Multicore Platforms

“…Temporal memory resource isolation: In 2020, Fang et al [42] proposed a memory access scheduling strategy to solve the problem of shared memory contention in systems where GPUs are used. The method consists of three steps: initially, the memory requests are separated into two queues in the memory controller, avoiding GPU memory access requests interfering with CPU requests.…”

“…However, fixed-size partitions cause low cache utilization and consequently reduced performance. On the other hand, in dynamic partitioning techniques, the size of Scheduling algorithms Temporal Hardware HRT [69] Memory request throttling Temporal Software SRT/HRT [74]- [77] Predictable DRAM Controllers Predictable Controller Hardware SRT/HRT [78], [81], [82] Bank Partitioning Spatial Software AVG [60] Channel Partitioning Spatial Soft/Hard AVG [80] Page Policy Control Spatial Soft/Hard AVG [83] Page Policy Control Spatial None AVG [84] Bank Partitioning Spatial None AVG [85] Decoupled Direct Access Spatial Hardware AVG [42] Scheduling algorithms Temporal None AVG [86] Task allocation Temporal Software AVG the allocated cache partitions varies during runtime, giving high cache utilization and causing lower predictability [98]. Furthermore, cache partitioning techniques can be characterized as index-based partitioning and way-based partitioning based on the structure of a set-associative cache.…”

Locality-Based Cache Management and Warp Scheduling for Reducing Cache Contention in GPU

“…Existing research uses multiple methods to manage GPU cache to solve the problem of the GPU memory subsystem and cache management efficiency. In addition to the above methods, there are warp throttling [32] and memory scheduling strategy [33].…”