A Survey Comparing Specialized Hardware And Evolution In TPUs For Neural Networks

Abstract

This survey paper is based on the evolution of TPUs from first generation TPUs to edge TPUs and their architectures. This paper compares CPUs, GPUs, FPGAs and TPUs, their hardware architectures, their similarities and differences will be discussed. Modern neural networks are immensely used these days but they require more time, computation and energy. Due to the greater demand and attractive options for architects to explore, companies are continuously working to reduce training and inference response time. Due to the demands and cost factors different kinds of ASICs (application specific integrated circuits) are developed and research is increased in this area. Many models of CPUs, GPUs and TPUs have been developed to support these networks and to improve training and inference phase. Intel developed CPUs for this purpose, NVIDIA developed GPUs and Google developed cloud TPUs. The hardware of CPUs and GPUs can be sold to businesses while Google offers TPU processing for everyone from the cloud. When the data is away from the computational source, it increases the overall cost and to reduce this cost companies implements memory management and caching techniques close to ALUs.

A Survey on Optimization Techniques for Edge Artificial Intelligence (AI)

“…Producers of TPU are Google, Coral, and Hailo. Further, the evolution of TPUs can be visualized in four generations [ 16 ]. The first-generation TPU is a CISC processor, and the complex instructions are executed by Matrix Multiplier Unit.…”

Advancements in Artificial Intelligence Circuits and Systems (AICAS)

“…TPUs demonstrate a notable reduction in energy consumption per computation, a critical factor in sustainable and scalable AI development. This efficiency is paramount in scenarios where large-scale computations are routine, making TPUs an indispensable asset in the AI and ML landscape [65,[81][82][83].…”

A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM

“…The trade-offs related to the realization of vision at the edge have been addressed from diverse perspectives. Specific and progressively more efficient devices have been developed in the last few years, such as low-power edge GPUs [1]- [3], neural-network accelerators [4], embedded CPUs [5], or tensor processor units [6]. Strategies based on multiple devices have also been proposed, including hybrid cloud-edge solutions [7]- [13] and collaborative systemsin the framework of the Internet of Things [14]- [18].…”