Cambricon: An Instruction Set Architecture for Neural Networks

Abstract

Neural Networks (NN) are a family of models for a broad range of emerging machine learning and pattern recondition applications. NN techniques are conventionally executed on general-purpose processors (such as CPU and GPGPU), which are usually not energy-efficient since they invest excessive hardware resources to flexibly support various workloads. Consequently, application-specific hardware accelerators for neural networks have been proposed recently to improve the energy-efficiency. However, such accelerators were designed for a small set of NN techniques sharing similar computational patterns, and they adopt complex and informative instructions (control signals) directly corresponding to high-level functional blocks of an NN (such as layers), or even an NN as a whole. Although straightforward and easy-to-implement for a limited set of similar NN techniques, the lack of agility in the instruction set prevents such accelerator designs from supporting a variety of different NN techniques with sufficient flexibility and efficiency. In this paper, we propose a novel domain-specific Instruction Set Architecture (ISA) for NN accelerators, called Cambricon, which is a load-store architecture that integrates scalar, vector, matrix, logical, data transfer, and control instructions, based on a comprehensive analysis of existing NN techniques. Our evaluation over a total of ten representative yet distinct NN techniques have demonstrated that Cambricon exhibits strong descriptive capacity over a broad range of NN techniques, and provides higher code density than general-purpose ISAs such as x86, MIPS, and GPGPU. Compared to the latest state-ofthe-art NN accelerator design DaDianNao [5] (which can only accommodate 3 types of NN techniques), our Cambricon-based accelerator prototype implemented in TSMC 65nm technology incurs only negligible latency/power/area overheads, with a versatile coverage of 10 different NN benchmarks.

eCNN

“…Instruction set. Previous SIMD works usually devised load instructions for parameters and adopted medium-grained operands for features, such as vector/matrix [37], 2D tile [49], and compute tile [47], for providing flexibility. In contrast, we apply a parameterinside approach and large-grained feature operands for optimizing power consumption and computing capability for highly-parallel convolution.…”

Newton: Gravitating Towards the Physical Limits of Crossbar Acceleration

“…Eyeriss [6] and ShiDianNao [9] improve the NFU dataflow to maximize operand reuse. A number of other digital designs [16], [20], [10] have also emerged in the past year. Analog Accelerators.…”

Design and Implementation of Deep Neural Network for Edge Computing

“…In addition, FT2000 and CEVA-XM6 are vector processors that includes vector processing unit and scalar processing unit, the main difference is that CEVA-XM6 is designed for accelerating only matrix convolution, while FT2000 is optimized by improvement of the algorithm. The similarity between FT2000 and Cambricon [22] are that they are all programmable by instruction set, using instruction set can quickly realize different kinds of neural network, except that Cambricon only simulates and does not have a slice. FT2000 and TPU Table 1 Comparison of parameters between FT2000 and current mainstream neural network accelerators are similar in their architecture, except that FT2000 is a general-purpose neural network accelerator, while TPU only supports CNN, LSTM, and MLP.…”

“…The computing time of convolutional layers accounts for about 85% of the total model [22], thus accelerating convolution calculation in CNN becomes a hotspot in current neural network acceleration. As the convolution calculation is mainly carried out by a large input feature map and a small convolutional kernel, the convolutional kernel is smaller in size, such as 1 × 1, 3 × 3, 5 × 5, whereas the input feature map has a larger scale, such as 224 × 224 × 3 in GoogLeNet.…”