PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning

Abstract

In this work, we present a reinforcement learning (RL) based approach to designing parallel prefix circuits such as adders or priority encoders that are fundamental to high-performance digital design. Unlike prior methods, our approach designs solutions tabula rasa purely through learning with synthesis in the loop. We design a grid-based state-action representation and an RL environment for constructing legal prefix circuits. Deep Convolutional RL agents trained on this environment produce prefix adder circuits that Pareto-dominate existing baselines with up to 16.0% and 30.2% lower area for the same delay in the 32b and 64b settings respectively. We observe that agents trained with open-source synthesis tools and cell library can design adder circuits that achieve lower area and delay than commercial tool adders in an industrial cell library.

General intelligence requires rethinking exploration

“…Recent deep SL methods perform highly accurate modelling across modalities like natural language [ 12 , 18 , 20 ], images [ 19 , 38 ] and speech [ 39 , 40 ]. In parallel, progress in deep RL has produced agents matching or beating top human players in strategic games like Go [ 41 , 42 ] and Stratego [ 43 ], while achieving state-of-the-art results in scientific pursuits like chip design [ 44 , 45 ] and controlling nuclear fusion plasma [ 46 ]. While impressive, these developments focus primarily on questions of model design and optimization.…”

ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design

“…Additionally, the diversity of the application landscape and the unique characteristics of the search space across the compute stack challenge the performance of conventional optimization methods. To address these challenges, both industry [70,85,98] and academia [37,41,51,52,86,89,97] have turned towards ML-driven optimization to meet stringent domain-specific requirements. Although prior work has demonstrated the benefits of ML in design optimization, the lack of reproducible baselines hinders fair and objective comparison across different methods.…”

New dictyopteran fossils from the Pennsylvanian of West Beijing, China

“…strategiesRoy et al (2021);Degrave et al (2022). However, this rapid development also presents challenges, as existing RL frameworks struggle to meet the community's growing demands, highlighting the need for an inclusive and adaptable framework.…”