CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework

Abstract

Recently, automated co-design of machine learning (ML) models and accelerator architectures has attracted significant attention from both the industry and academia. However, most co-design frameworks either explore a limited search space or employ suboptimal exploration techniques for simultaneous design decision investigations of the ML model and the accelerator. Furthermore, training the ML model and simulating the accelerator performance is computationally expensive. To address these limitations, this work proposes a novel neural architecture and hardware accelerator co-design framework, called CODEBench. It comprises two new benchmarking sub-frameworks, CNNBench and AccelBench, which explore expanded design spaces of convolutional neural networks (CNNs) and CNN accelerators. CNNBench leverages an advanced search technique, BOSHNAS, to efficiently train a neural heteroscedastic surrogate model to converge to an optimal CNN architecture by employing second-order gradients. AccelBench performs cycle-accurate simulations for diverse accelerator architectures in a vast design space. With the proposed co-design method, called BOSHCODE, our best CNN-accelerator pair achieves 1.4% higher accuracy on the CIFAR-10 dataset compared to the state-of-the-art pair while enabling 59.1% lower latency and 60.8% lower energy consumption. On the ImageNet dataset, it achieves 3.7% higher Top1 accuracy at 43.8% lower latency and 11.2% lower energy consumption. CODEBench outperforms the state-of-the-art framework, i.e., Auto-NBA, by achieving 1.5% higher accuracy and 34.7 × higher throughput while enabling 11.0 × lower energy-delay product (EDP) and 4.0 × lower chip area on CIFAR-10.

Generic neural architecture search toolkit for efficient and real-world deployment of visual inspection convolutional neural networks in industry

“…The same authors further extended their work and created FBNetV2, 55 a new approach able to optimize both spatial (height and width) and channel dimensions of layers in CNNs while being more memory-efficient and incurring lower computational costs compared to its predecessor. Other methods worth mentioning here are ProxylessNAS 56 and FLASH 57 …”

AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers

“…The LP mode for AccelTran-Edge is also considered. proach [52] could also efficiently test different buffer sizes, along with the corresponding ratios that may be optimal for each transformer model. We defer this automated co-design method to future work.…”

Automatic Design of a Broadband Directional Coupler via Bayesian Optimization

“…The second line of work uses BO to speed up the design of deep neural network (DNN) accelerators [240], [246]- [249]. DNNs have gained significant attention due to their successes in various areas at the expense of high computation and memory cost.…”

“…The design of a DNN accelerator typically involves hardware (e.g., custom circuits for matrix multiplication) and software (e.g., efficient data-parallel processing algorithm) co-design. The co-design problem is framed as an optimization in the joint space of hardware architectures and software components, and BO has been demonstrated to achieve satisfactory results in this context [240], [246]- [249].…”