Core Placement Optimization for Multi-chip Many-core Neural Network Systems with Reinforcement Learning

Abstract

Multi-chip many-core neural network systems are capable of providing high parallelism benefited from decentralized execution, and they can be scaled to very large systems with reasonable fabrication costs. As multi-chip many-core systems scale up, communication latency related effects will take a more important portion in the system performance. While previous work mainly focuses on the core placement within a single chip, there are two principal issues still unresolved: the communication-related problems caused by the non-uniform, hierarchical on/off-chip communication capability in multi-chip systems, and the scalability of these heuristic-based approaches in a factorially growing search space. To this end, we propose a reinforcement-learning-based method to automatically optimize core placement through deep deterministic policy gradient, taking into account information of the environment by performing a series of trials (i.e., placements) and using convolutional neural networks to extract spatial features of different placements. Experimental results indicate that compared with a naive sequential placement, the proposed method achieves 1.99× increase in throughput and 50.5% reduction in latency; compared with the simulated annealing, an effective technique to approximate the global optima in an extremely large search space, our method improves the throughput by 1.22× and reduces the latency by 18.6%. We further demonstrate that our proposed method is capable to find optimal placements taking advantages of different communication properties caused by different system configurations, and work in a topology-agnostic manner. CCS Concepts: • Computer systems organization → Architectures; Parallel architectures; • Computing methodologies → Reinforcement learning;

A Survey of Machine Learning for Computer Architecture and Systems

“…Targeting the heat interaction of processor cores and NoC routers, Lu et al [147] apply Q-learning to perform task assignments to specific cores based on current temperatures of cores and routers, such that the maximum temperature in the future is minimized. Targeting the non-uniform and hierarchical on/off-chip communication capability in multi-chip many-core systems, core placement optimization [242] leverages deep deterministic policy gradient (DDPG) [140] to map computation onto physical cores, able to work in a manner agnostic to domain-specific information.…”

Continual Learning Approach for Improving the Data and Computation Mapping in Near-Memory Processing System

“…It has also been used for designing memory systems, such as prefetching [52] and memory controller [53]. Additionally, it has been applied to DNN compilation and mapping optimization [54,55,56]. In this work, we use RL for co-exploration of data and computation mapping in NMP systems.…”

Approaching the mapping limit with closed-loop mapping strategy for deploying neural networks on neuromorphic hardware

“…To overcome the memory wall problem, the decentralized manycore architecture emerges in recent years for performing neural network workloads, which presents massive processing parallelism, memory locality, and multi-chip scalability (Painkras et al, 2013 ; Akopyan et al, 2015 ; Han et al, 2016 ; Jouppi et al, 2017 ; Parashar et al, 2017 ; Shin et al, 2017 ; Davies et al, 2018 ; Chen et al, 2019 ; Pei et al, 2019 ; Shao et al, 2019 ; Deng et al, 2020 ; Zimmer et al, 2020 ). Each functional core contains independent computation and memory resources with close distance, and cores communicate through a flexible routing fabric (Wu et al, 2020 ). Due to the limited hardware resources in each core, a large neural network model has to be partitioned and mapped onto cores during deployment.…”

“…In the logical mapping stage, the requirements for computation and memory resources are important consideration factors for allocating cores. The parameters and the associated computations are divided into small slices through tensor dimension partition and each slice is allocated into a single core with limited hardware resources (Shao et al, 2019 ; Deng et al, 2020 ; Wu et al, 2020 ). For a convolutional layer, most previous work adopts the 2D partition to split the input channel ( C in ) and the output channel ( C out ) dimensions.…”

“…This stage does not care the physical locations of cores on real hardware. The physical mapping places each logical core to a physical location in the chip, which greatly affects the communication latency and might cause the deadlock problem (Wu et al, 2020 ). The core placement optimization for minimized latency is actually an NP-hard problem (Myung et al, 2021 ) and the search space grows rapidly as the number of cores increases.…”