INFER: INterFerence-aware Estimation of Runtime for Concurrent CNN Execution on DPUs

Goel, Shikha; Kedia, Rajesh; Balakrishnan, M.; Sen, Rijurekha

doi:10.1109/icfpt51103.2020.00018

Cited by 10 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While smart cameras, with specialized compute capabilities to run DNN's are available in the market, they cost at least 5× more than conventional cameras [15]. Thus, current surveillance techniques either depend on heuristics as proposed by DDS or Reducto, or utilize specialized hardware such as FPGA's [16]. While these techniques are feasible, they also increase the complexity and cost of the system.…”

Section: B Shortcomings Of Existing Traffic Surveillance Systemsmentioning

confidence: 99%

VISTA: Fast and Efficient Traffic Surveillance by Tile Sampling

Chaudhary¹,

Taneja²,

Singh³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>With the increasing number of vehicles in modern cities, traffic surveillance via cameras on roads has become an important application. Cities have installed thousands of cameras on roads, which send video feeds to a cloud center to run computer vision algorithms. This requires high bandwidth. Current techniques reduce the bandwidth requirement by either sending a limited number of frames/pixels/regions or relying on re-encoding the important parts of the video. This requires running DNNs to extract important portions in a frame so that they can be again sent at a higher resolution from the camera to the server. This has the disadvantage of imposing significant overhead on the camera side compute, as re-encoding is known to be expensive, and makes the system less real-time. In this work, we propose VISTA, a system that utilizes tile sampling, where a limited number of rectangular areas within the frames, known as tiles, are sent to the server. We then propose an adaptive tile sampling algorithm, that estimates the presence of moving objects by comparing the statistics of the tiles' bitrate (in kbps) and then decide to retain only the necessary tiles, thus eliminating the requirement to use a DNN at the camera side. We evaluate VISTA on different datasets having 56 videos in total to show that on average our technique reduces $17$-$40$\% of the total amount of data sent to the cloud while providing a detection accuracy of over $85\%$. Furthermore, VISTA also runs in real-time even on cheap edge devices like Raspberry Pi and nVidia Jetson Nano. Further, it requires minimal calibration compared to prior works.</p>

show abstract

Section: B Shortcomings Of Existing Traffic Surveillance Systemsmentioning

confidence: 99%

VISTA: Fast and Efficient Traffic Surveillance by Tile Sampling

Chaudhary¹,

Taneja²,

Singh³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…3: System software for multi-DNN systems. through design space exploration [77], [82] and contention-aware performance estimation techniques [83].…”

Section: A Multi-tenant Ai Systemsmentioning

confidence: 99%

How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures

Venieris

Panopoulos

Leontiadis

et al. 2021

2021 IEEE 32nd International Conference on Application-Specific Systems, Architectures and Processors (ASAP)

View full text Add to dashboard Cite

The unprecedented performance of deep neural networks (DNNs) has led to large strides in various Artificial Intelligence (AI) inference tasks, such as object and speech recognition. Nevertheless, deploying such AI models across commodity devices faces significant challenges: large computational cost, multiple performance objectives, hardware heterogeneity and a common need for high accuracy, together pose critical problems to the deployment of DNNs across the various embedded and mobile devices in the wild. As such, we have yet to witness the mainstream usage of state-of-the-art deep learning algorithms across consumer devices. In this paper, we provide preliminary answers to this potentially game-changing question by presenting an array of design techniques for efficient AI systems. We start by examining the major roadblocks when targeting both programmable processors and custom accelerators. Then, we present diverse methods for achieving real-time performance following a cross-stack approach. These span model-, system-and hardware-level techniques, and their combination. Our findings provide illustrative examples of AI systems that do not overburden mobile hardware, while also indicating how they can improve inference accuracy. Moreover, we showcase how custom ASIC-and FPGA-based accelerators can be an enabling factor for next-generation AI applications, such as multi-DNN systems. Collectively, these results highlight the critical need for further exploration as to how the various cross-stack solutions can be best combined in order to bring the latest advances in deep learning close to users, in a robust and efficient manner.

show abstract

“…In terms of flexible deployment of Xilinx DPUs, Rajesh Kediad et al studied the design space exploration of DPU based on a FPGA platform [18], demonstrating the feasibility and flexibility of different DPU deployments. Furthermore, they estimated the energy consumption of different CNN models on the DPU platform [19]. Rajesh Kediad's research motivated us to use DPUs for adaptive reconfiguration.…”

Section: Introductionmentioning

confidence: 99%

Feasibility Analysis and Implementation of Adaptive Dynamic Reconfiguration of CNN Accelerators

Han

Luo

2022

Electronics

View full text Add to dashboard Cite

In multi-tasking scenarios with dynamically changing loads, the parallel computing of convolutional neural networks (CNNs) causes high energy and resource consumption in the system. Another critical problem is that previous neural network hardware accelerators are often limited to fixed scenarios and lack the function of adaptive adjustment. To solve these problems, a reconfiguration adaptive system based on the prediction of algorithm workload is proposed in this paper. Deep Learning Processor Unit (DPU) from Xilinx has excellent performance in accelerating network computing. After summarizing the characteristics of hardware accelerators and gaining an in-depth understanding of the DPU structure, we propose a regression model for CNNs runtime prediction and a guidance scheme for adaptive reconfiguration combined with the characteristics of Deep Learning Processor Unit. For different DPU sizes, the accuracy of the proposed prediction model achieves 90.7%. With the dynamic reconfiguration technology, the proposed strategy can enable accurate and fast reconfiguration. In the load change scenario, the proposed system can significantly reduce power consumption.

show abstract

INFER: INterFerence-aware Estimation of Runtime for Concurrent CNN Execution on DPUs

Cited by 10 publications

References 11 publications

VISTA: Fast and Efficient Traffic Surveillance by Tile Sampling

VISTA: Fast and Efficient Traffic Surveillance by Tile Sampling

How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures

Feasibility Analysis and Implementation of Adaptive Dynamic Reconfiguration of CNN Accelerators

Contact Info

Product

Resources

About