A Survey of Deep Learning on CPUs: Opportunities and Co-Optimizations

Mittal, Sparsh; Rajput, Poonam; Subramoney, Sreenivas

doi:10.1109/tnnls.2021.3071762

Cited by 39 publications

(11 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In academia, while many purpose-built accelerators [6,9,24,46] have been proposed, including recent efforts targeting sparsity [18], they pay insufficient attention to software integration or rapid workload evolution, leading to quick obsolescence, alluded to by [26]. Mittal et al [39] presents a survey of deep-learning on CPUs where the SW lift is ameliorated somewhat. Cambricon [38] is of interest, since it exposes linear algebra concepts as architecturally supported primitives -similar to how RED exposes new primitives at the ISA level, but lacks a performance contract between the implementation and programmer.…”

Section: Related Workmentioning

confidence: 99%

The Mozart reuse exposed dataflow processor for AI and beyond

Sankaralingam

Nowatzki

Gangadhar

et al. 2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

In this paper we introduce the Mozart Processor, which implements a new processing paradigm called Reuse Exposed Dataflow (RED). RED is a counterpart to existing execution models of Von-Neumann, SIMT, Dataflow, and FPGA. Dataflow and data reuse are the fundamental architecture primitives in RED, implemented with mechanisms for inter-worker communication and synchronization. The paper defines the processor architecture, the details of the microarchitecture, chip implementation, software stack development, and performance results. The architecture's goal is to achieve near-CPU like flexibility while having ASIC-like efficiency for a large-class of data-intensive workloads. An additional goal was software maturity -have large coverage of applications immediately, avoiding the need for a long-drawn hand-tuning software development phase. The architecture was defined with this software-maturity/compiler friendliness in mind. In short, the goal was to do to GPUs, what GPUs did to CPUs -i.e. be a better solution for a large range of workloads, while preserving flexibility and programmability. The chip was implemented with HBM and PCIe interfaces and taken to production on a 16nm TSMC FFC process. For ML inference tasks with batch-size=4, Mozart is integer factors better than state-of-theart GPUs even while being nearly 2 technology nodes behind. We conclude with a set of lessons learned, the unique challenges of a clean-slate architecture in a commercial setting, and pointers for uncovered research problems. CCS CONCEPTS• Computer systems organization → Data flow architectures;• Hardware → Hardware accelerators.

show abstract

Section: Related Workmentioning

confidence: 99%

The Mozart reuse exposed dataflow processor for AI and beyond

Sankaralingam

Nowatzki

Gangadhar

et al. 2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

show abstract

“…6), it seems that SDG is not the only tool that can help avoid saddle points. Uncertainty in GPUs may also act as a tool for avoiding saddle points, although we have not been able to know this until now because neural networks are rarely studied on CPUs (Mittal et al, 2021). Training was performed under condition 3 (cuDNN = on; initial value = fixed) using CPU (i7 6700K) or GPU (Titan RTX).…”

Section: Uncertainty and Deep Neural Networkmentioning

confidence: 99%

Impact of GPU uncertainty on the training of predictive deep neural networks

Pietrowski¹,

Gajda²,

Yamamoto³

et al. 2021

Preprint

View full text Add to dashboard Cite

retracted] We found out that the difference was dependent on the Chainer library, and does not replicate with another library (PyTorch) which indicates that the results are probably due to a bug in Chainer, rather than being hardware-dependent.-old abstract Deep neural networks often present uncertainties such as hardwareand software-derived noise and randomness. We studied the effects of such uncertainty on learning outcomes, with a particular focus on the function of graphics processing units (GPUs), and found that GPU-induced uncertainty increased learning accuracy of a certain deep neural network. When training a predictive deep neural network using only the CPU without the GPU, the learning error is higher than when training the same number of epochs using the GPU, suggesting that the GPU plays a different role in the learning process than just increasing the computational speed. Because this effect cannot be observed in learning by a simple autoencoder, it could be a phenomenon specific to certain types of neural networks. GPU-specific computational processing is more indeterminate than that by CPUs, and hardware-derived uncertainties, which are often considered obstacles that need to be eliminated, might, in some cases, be successfully incorporated into the training of deep neural networks. Moreover, such uncertainties might be interesting phenomena to consider in brain-related computational processing, which comprises a large mass of uncertain signals.

show abstract

“…Although GPUs are the main hardware platforms in deep learning fields, there are many factors to motivate CNNs running on resource-constrained systems including mobile devices (computational and energy constraints) and CPU-based servers (computational constraints relative to popular GPUs) [18]. In mobile computing systems, CPUs maybe perform better than GPU in terms of performance and power consumption.…”

Section: Introductionmentioning

confidence: 99%

Towards Effective Depthwise Convolutions on ARMv8 Architecture

Hao¹,

Wang²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Depthwise convolutions are widely used in lightweight convolutional neural networks (CNNs). The performance of depthwise convolutions is mainly bounded by the memory access rather than the arithmetic operations for classic convolutions so that direct algorithms are often more efficient than indirect ones (matrix multiplication-, Winograd-, and FFT-based convolutions) with additional memory accesses. However, the existing direct implementations of depthwise convolutions on ARMv8 architectures feature a bad tradeoff between register-level reuse of different tensors, which usually leads to sub-optimal performance. In this paper, we propose new direct implementations of depthwise convolutions by means of implicit padding, register tiling, etc., which contain forward propagation, backward propagation and weight gradient update procedures. Compared to the existing ones, our new implementations can incur much less communication overhead between registers and cache. Experimental results on two ARMv8 CPUs show that our implementations can averagely deliver 4.88x and 16.4x performance improvement over the existing direct ones in open source libraries and matrix multiplications-based ones in Pytorch, respectively.

show abstract

A Survey of Deep Learning on CPUs: Opportunities and Co-Optimizations

Cited by 39 publications

References 73 publications

The Mozart reuse exposed dataflow processor for AI and beyond

The Mozart reuse exposed dataflow processor for AI and beyond

Impact of GPU uncertainty on the training of predictive deep neural networks

Towards Effective Depthwise Convolutions on ARMv8 Architecture

Contact Info

Product

Resources

About