End-to-End Synthesis of Dynamically Controlled Machine Learning Accelerators

Abstract

Edge systems are required to autonomously make real-time decisions based on large quantities of input data under strict power, performance, area, and other constraints. Meeting these constraints is only possible by specializing systems through hardware accelerators purposefully built for machine learning and data analysis algorithms. However, data science evolves at a quick pace, and manual design of custom accelerators has high non-recurrent engineering costs: general solutions are needed to automatically and rapidly transition from the formulation of a new algorithm to the deployment of a dedicated hardware implementation. Our solution is the SOftware Defined Architectures (SODA) Synthesizer, an end-to-end, multi-level, modular, extensible compiler toolchain providing a direct path from machine learning tools to hardware. The SODA Synthesizer frontend is based on the multilevel intermediate representation (MLIR) framework; it ingests pre-trained machine learning models, identifies kernels suited for acceleration, performs high-level optimizations, and prepares them for hardware synthesis. In the backend, SODA leverages state-of-the-art high-level synthesis techniques to generate highly efficient accelerators, targeting both field programmable devices (FPGAs) and applicationspecific circuits (ASICs). In this paper, we describe how the SODA Synthesizer can also assemble the generated accelerators (based on the finite state machine with datapath model) in a custom system driven by a distributed controller, building a coarse-grained dataflow architecture that does not require a host processor to orchestrate parallel execution of multiple accelerators. We show the effectiveness of our approach by automatically generating ASIC accelerators for layers of popular deep neural networks (DNNs). Our high-level optimizations result in up to 74x speedup on isolated accelerators for individual DNN layers, and our dynamically scheduled architecture yields an additional 3x performance improvement when combining accelerators to handle streaming inputs.

HERMES: qualification of High pErformance pRogrammable Microprocessor and dEvelopment of Software ecosystem

“…The HERMES project use cases include applications based on artificial intelligence, which might contain multiple parallel execution flows (i.e., coarse-grained parallelism); when synthesized through an HLS tool, the complexity of the finite state machine controllers for such applications grows exponentially, leading to considerable resource consumption and latency overheads. To solve this problem, Bambu has been extended to efficiently synthesize dynamically controlled accelerators and integrated in a compiler-based toolchain that facilitates the extraction of coarse-grained tasks and data dependencies from an input machine learning application designed and trained in a high-level programming framework [14]. Future developments will focus on other optimization techniques and architectural templates that can answer the specific needs of artificial intelligence algorithms and the requirements of aerospace applications.…”

A Flexible and General-Purpose Platform for Heterogeneous Computing

“…Most parallel computing approaches optimize implementation for a single device using a specific implementation framework: for example, Python/HLS for FPGAs [6], Python/CUDA for GPUs [7], or C/HLS for ASICs/FPGAs [8,9]. Ad hoc solutions do not allow changes to the originally defined computing architecture to support algorithms other than those for which they were created.…”

“…All of these devices are used simultaneously. This could result in a more flexible computation approach than developing a solution for a specific device using a custom design flow, as presented in [6][7][8][9]; the main advantage of heterogeneous computing lies in its flexibility and generality, possibly at the cost of performance.…”