A Deep Reinforcement Learning Framework for Architectural Exploration: A Routerless NoC Case Study

Lin, Ting-Ru; Penney, Drew; Pedram, Massoud; Chen, Lizhong

doi:10.1109/hpca47549.2020.00018

Cited by 34 publications

(21 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, Pythia incurs low hardware overhead. Researchers have also explored ML techniques to explore the large microarchitectural design space, e.g., NoC design [39,43,44,50,54,83,127,135,136,142], chip placement optimization [90], hardware resource assignment for accelerators [75]. These works are orthogonal to Pythia.…”

Section: Other Related Workmentioning

confidence: 99%

Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

Bera

Kanellopoulos

Nori

et al. 2021

MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address, or delta between cacheline addresses) to predict future memory accesses. These techniques either completely neglect a prefetcher's undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm. We show that prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design.To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and systemaware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms two state-of-the-art prefetchers (MLOP and Bingo) by 3.4% and 3.8% in single-core, 7.7% and 9.6% in twelve-core, and 16.9% and 20.2% in bandwidth-constrained core configurations, while incurring only 1.03% area overhead over a desktop-class processor and no software changes in workloads. The source code of Pythia can be freely downloaded from https://github.com/CMU-SAFARI/Pythia.

show abstract

Section: Other Related Workmentioning

confidence: 99%

Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

Bera

Kanellopoulos

Nori

et al. 2021

MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

show abstract

“…However, the exploration of the environment is not likely to exhaust all states though the action policy contains random actions or noise [44,45], which leads to the global optimum may being buried in the quagmire. In particular, the proposed communication scenarios and indoor layouts have high complexity, impelling the efficient and sufficient exploration to be a problem.…”

Section: Federated Learning Modelmentioning

confidence: 99%

Mobile Reconfigurable Intelligent Surfaces for NOMA Networks: Federated Learning Approaches

Zhong¹,

Liu²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

A novel framework of reconfigurable intelligent surfaces (RISs)-enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enable mobility of the RIS and enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable to provide NOMA with artificial controlled channel conditions, which can be seen as a beneficial operation condition to obtain NOMA gains. To optimize the sum rate of all users, a deep deterministic policy gradient (DDPG) algorithm is invoked to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. In order to improve the efficiency and effectiveness of agent training for the DDPG agents, a federated learning (FL) concept is adopted to enable multiple agents to simultaneously explore similar environments and exchange experiences. We also proved that with the same random exploring policy, the FL armed deep reinforcement learning (DRL) agents can theoretically obtain a reward gain compare to the independent agents. Our simulation results indicate that the mobile RIS scheme can significantly outperform the fixed RIS paradigm, which provides about three times data rate gain compare to the fixed RIS paradigm. Moreover, the NOMA scheme is capable to achieve a gain of 42% in contrast with the OMA scheme in terms of sum rate. Finally, the multi-cell simulation proved that the FL enhanced DDPG algorithm has a superior convergence rate and optimization performance than the independent training framework.Index terms-Deep reinforcement learning (DRL), federated learning (FL), intelligent reflecting surfaces (IRSs), non-orthogonal multiple access (NOMA), reconfigurable intelligent surfaces (RIS), resource management

show abstract

“…Ring Network. The ring network has been adopted in real multi-processors design [11] (e.g., Intel Xeon-Phi [23]) and logically studied for efficient collective communication [8], [10], [44], [45], [46], [72]. On the one hand, the ring network provide simple routing mechanism to better-utilized the link bandwidth for fast communication [11].…”

Section: Related Workmentioning

confidence: 99%

“…on the other hand, the ring network has been criticized for easy saturation due to linear increased latency with more node [7], [35], [46]. Previous works [8], [45], [46], [72] extend the ring concept to form local-rings network among a subset of nodes, known as routerless network. In ARENA, we avoid the saturation problem through: (a) a routerless task execution model among nodes; and (b) the dynamically task allocation and dispatching mechanism.…”

Section: Related Workmentioning

confidence: 99%

ARENA: Asynchronous Reconfigurable Accelerator Ring to Enable Data-Centric Parallel Computing

Tan

Xie

Geng

et al. 2021

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

The next generation HPC and data centers are likely to be reconfigurable and data-centric due to the trend of hardware specialization and the emergence of data-driven applications. In this paper, we propose ARENA -an asynchronous reconfigurable accelerator ring architecture as a potential scenario on how the future HPC and data centers will be like. Despite using the coarse-grained reconfigurable arrays (CGRAs) as the substrate platform, our key contribution is not only the CGRA-cluster design itself, but also the ensemble of a new architecture and programming model that enables asynchronous tasking across a cluster of reconfigurable nodes, so as to bring specialized computation to the data rather than the reverse. We presume distributed data storage without asserting any prior knowledge on the data distribution. Hardware specialization occurs at runtime when a task finds the majority of data it requires are available at the present node. In other words, we dynamically generate specialized CGRA accelerators where the data reside. The asynchronous tasking for bringing computation to data is achieved by circulating the task token, which describes the dataflow graphs to be executed for a task, among the CGRA cluster connected by a fast ring network. Evaluations on a set of HPC and data-driven applications across different domains show that ARENA can provide better parallel scalability with reduced data movement (53.9%). Compared with contemporary compute-centric parallel models, ARENA can bring on average 4.37× speedup. The synthesized CGRAs and their task-dispatchers only occupy 2.93mm 2 chip area under 45nm process technology and can run at 800MHz with on average 759.8mW power consumption. ARENA also supports the concurrent execution of multi-applications, offering ideal architectural support for future high-performance parallel computing and data analytics systems.

show abstract

A Deep Reinforcement Learning Framework for Architectural Exploration: A Routerless NoC Case Study

Cited by 34 publications

References 39 publications

Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

Mobile Reconfigurable Intelligent Surfaces for NOMA Networks: Federated Learning Approaches

ARENA: Asynchronous Reconfigurable Accelerator Ring to Enable Data-Centric Parallel Computing

Contact Info

Product

Resources

About