Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review

Shuvo, Md. Maruf Hossain; Islam, Syed K.; Cheng, Jianlin; Morshed, Bashir I.

doi:10.1109/jproc.2022.3226481

Cited by 78 publications

(10 citation statements)

References 380 publications

(341 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By leveraging techniques like compound scaling, which uniformly scales the network width, depth, and resolution, EfficientNet optimizes the model’s architecture to maximize accuracy while minimizing the number of parameters and computations. This enables real-time inference and efficient utilization of resources on edge devices, ensuring faster and more responsive image processing capabilities even with limited computing power [ 56 ]. Moreover, in the considered application, the input size of the available pretrained EfficientNet B5 models matches the resolution of our target images.…”

Section: Methodsmentioning

confidence: 99%

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices

Rostkowska

Skrzypczyński

2023

Sensors

View full text Add to dashboard Cite

This paper considers the task of appearance-based localization: visual place recognition from omnidirectional images obtained from catadioptric cameras. The focus is on designing an efficient neural network architecture that accurately and reliably recognizes indoor scenes on distorted images from a catadioptric camera, even in self-similar environments with few discernible features. As the target application is the global localization of a low-cost service mobile robot, the proposed solutions are optimized toward being small-footprint models that provide real-time inference on edge devices, such as Nvidia Jetson. We compare several design choices for the neural network-based architecture of the localization system and then demonstrate that the best results are achieved with embeddings (global descriptors) yielded by exploiting transfer learning and fine tuning on a limited number of catadioptric images. We test our solutions on two small-scale datasets collected using different catadioptric cameras in the same office building. Next, we compare the performance of our system to state-of-the-art visual place recognition systems on the publicly available COLD Freiburg and Saarbrücken datasets that contain images collected under different lighting conditions. Our system compares favourably to the competitors both in terms of the accuracy of place recognition and the inference time, providing a cost- and energy-efficient means of appearance-based localization for an indoor service robot.

show abstract

Section: Methodsmentioning

confidence: 99%

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices

Rostkowska

Skrzypczyński

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The review performed in [6] provides a comprehensive examination of tools and techniques for efficient edge inference, a key element in AI on edge devices. It discusses the challenges of deploying computationally expensive and power-hungry DL algorithms in end-user applications, especially on resource-constrained devices like mobile phones and wearables.…”

Section: Related Workmentioning

confidence: 99%

Reducing the Power Consumption of Edge Devices Supporting Ambient Intelligence Applications

Fanariotis,

Orphanoudakis,

Fotopoulos

2024

Information

View full text Add to dashboard Cite

Having as a main objective the exploration of power efficiency of microcontrollers running machine learning models, this manuscript contrasts the performance of two types of state-of-the-art microcontrollers, namely ESP32 with an LX6 core and ESP32-S3 with an LX7 core, focusing on the impact of process acceleration technologies like cache memory and vectoring. The research employs experimental methods, where identical machine learning models are run on both microcontrollers under varying conditions, with particular attention to cache optimization and vector instruction utilization. Results indicate a notable difference in power efficiency between the two microcontrollers, directly linked to their respective process acceleration capabilities. The study concludes that while both microcontrollers show efficacy in running machine learning models, ESP32-S3 with an LX7 core demonstrates superior power efficiency, attributable to its advanced vector instruction set and optimized cache memory usage. These findings provide valuable insights for the design of power-efficient embedded systems supporting machine learning for a variety of applications, including IoT and wearable devices, ambient intelligence, and edge computing and pave the way for future research in optimizing machine learning models for low-power, embedded environments.

show abstract

“…Recent advances in Edge computing allow artificial intelligence and other computations to be performed onboard the device. These computations can be real-time and run on resource-constrained platforms, thus reducing latency and power consumption and addressing privacy-related issues [71]. Still, computationally intensive tasks such as medical imaging that do not need real-time processing can be performed over cloud services.…”

Section: Considerationsmentioning

confidence: 99%

Artificial intelligence enabled smart digital eye wearables

RaviChandran,

Teo,

Ting

2023

Current Opinion in Ophthalmology

View full text Add to dashboard Cite

Purpose of review Smart eyewear is a head-worn wearable device that is evolving as the next phase of ubiquitous wearables. Although their applications in healthcare are being explored, they have the potential to revolutionize teleophthalmology care. This review highlights their applications in ophthalmology care and discusses future scope. Recent findings Smart eyewear equips advanced sensors, optical displays, and processing capabilities in a wearable form factor. Rapid technological developments and the integration of artificial intelligence are expanding their reach from consumer space to healthcare applications. This review systematically presents their applications in treating and managing eye-related conditions. This includes remote assessments, real-time monitoring, telehealth consultations, and the facilitation of personalized interventions. They also serve as low-vision assistive devices to help visually impaired, and can aid physicians with operational and surgical tasks. Summary Wearables such as smart eyewear collects rich, continuous, objective, individual-specific data, which is difficult to obtain in a clinical setting. By leveraging sophisticated data processing and artificial intelligence based algorithms, these data can identify at-risk patients, recognize behavioral patterns, and make timely interventions. They promise cost-effective and personalized treatment for vision impairments in an effort to mitigate the global burden of eye-related conditions and aging.

show abstract

Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review

Cited by 78 publications

References 380 publications

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices

Reducing the Power Consumption of Edge Devices Supporting Ambient Intelligence Applications

Artificial intelligence enabled smart digital eye wearables

Contact Info

Product

Resources

About