GANAX: A Unified MIMD-SIMD Acceleration for Generative Adversarial Networks

Abstract

Generative Adversarial Networks (GANs) are one of the most recent deep learning models that generate synthetic data from limited genuine datasets. GANs are on the frontier as further extension of deep learning into many domains (e.g., medicine, robotics, content synthesis) requires massive sets of labeled data that is generally either unavailable or prohibitively costly to collect. Although GANs are gaining prominence in various fields, there are no accelerators for these new models. In fact, GANs leverage a new operator, called transposed convolution, that exposes unique challenges for hardware acceleration. This operator first inserts zeros within the multidimensional input, then convolves a kernel over this expanded array to add information to the embedded zeros. Even though there is a convolution stage in this operator, the inserted zeros lead to underutilization of the compute resources when a conventional convolution accelerator is employed. We propose the GANAX architecture to alleviate the sources of inefficiency associated with the acceleration of GANs using conventional convolution accelerators, making the first GAN accelerator design possible. We propose a reorganization of the output computations to allocate compute rows with similar patterns of zeros to adjacent processing engines, which also avoids inconsequential multiply-adds on the zeros. This compulsory adjacency reclaims data reuse across these neighboring processing engines, which had otherwise diminished due to the inserted zeros. The reordering breaks the full SIMD execution model, which is prominent in convolution accelerators. Therefore, we propose a unified MIMD-SIMD design for GANAX that leverages repeated patterns in the computation to create distinct microprograms that execute concurrently in SIMD mode. The interleaving of MIMD and SIMD modes is performed at the granularity of single microprogrammed operation. To amortize the cost of MIMD execution, we propose a decoupling of data access from data processing in GANAX. This decoupling leads to a new design that breaks each processing engine to an access micro-engine and an execute micro-engine. The proposed architecture extends the concept of access-execute architectures to the finest granularity of computation for each individual operand. Evaluations with six GAN models shows, on average, 3.6× speedup and 3.1× energy savings over EYERISS without compromising the efficiency of conventional convolution accelerators. These benefits come with a mere ≈7.8% area increase. These results suggest that GANAX is an effective initial step that paves the way for accelerating the next generation of deep neural models.

Asv

“…We observe that stereo DNNs make heavy use of the deconvolution operation 1 that exposes specific kernel sparsity, making conventional DNN accelerators inefficient. While prior work proposed specialized hardware to exploit deconvolution sparsity [60,76], we demonstrate that static software optimizations achieve better results without unnecessary hardware modifications.…”

“…Stereo vision DNNs make use of deconvolution layers, which expose structured sparsity patterns. Recent work has prosed specialized hardware specifically for exploiting sparsity in deconvolution layers [60,76]. Our observation, however, is that mitigating sparsityinduced efficiencies in deconvolution does not necessarily require hardware support.…”

“…Due to the various patterns needed to generate different output elements, deconvolution is clearly an "irregular" operation. Prior work [76] exploits the four…”

Towards a Uniform Architecture for the Efficient Implementation of 2D and 3D Deconvolutional Neural Networks on FPGAs

“…Few works have focused on accelerating deconvolutions [6,11,12]. In [11,12], the researchers addressed the accelerations of the deconvolution in generative adversarial networks(GANs).…”

Optimizing GPU Cache Policies for MI Workloads

“…The initial catalyst for this rise in popularity was the discovery that MI could produce low error rates for image classification [1][2] [3]. Subsequently, there has been a large amount of work optimizing hardware for MI, especially for Convolutional Neural Networks (CNNs) (e.g., [17]- [30]). Although these works have led to significant improvements in performance and energy efficiency of CNNs on modern multi-core CPUs, GPUs, and accelerators, it is challenging to analyze how future architectures will perform for these workloads.…”