Accelerating Deep Neural Networks implementation: A survey

Dhouibi, Meriam; Salem, Ahmed Karim Ben; Saidi, Afef; Saoud, Slim Ben

doi:10.1049/cdt2.12016

Cited by 22 publications

(6 citation statements)

References 92 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research on accelerating deep neural network (DNN) computation is receiving increased attention, especially as DNNs exhibit exceptional performance across various artificial intelligence disciplines. [ 18 ] A primary challenge in contemporary DNN acceleration lies in the extensive data movement between on‐chip processors and off‐chip memory. [ 19 ] In contrast, the binary neural network (BNN) is garnering interest due to its significantly reduced memory requirements (Figure 1a‐i).…”

Section: Resultsmentioning

confidence: 99%

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Kumar,

Krishnaiah,

Park

et al. 2023

Adv Funct Materials

View full text Add to dashboard Cite

In the realm of neuromorphic computing, integrating Binary Neural Networks (BNN) with non‐volatile memory based on emerging materials can be a promising avenue for introducing novel functionalities. This study underscores the viability of lead‐free, air‐stable Cs2SnI6 (CSI) based resistive random access memory (RRAM) devices as synaptic weights in neuromorphic architectures, specifically for BNNs applications. Herein, hydrothermally synthesized CSI perovskites are explored as a resistive layer in RRAM devices either on the rigid or flexible substrate, highlighting reproducible multibit switching with self‐compliance, low‐ resistance‐state (LRS) variations, a decent On/Off ratio(or retention) of ≈103(or 104 s), and endurance exceeding 300 cycles. Moreover, a comprehensive evaluation with the 32 × 32 × 3 RGB CIFAR‐10 dataset reveals that binary convolutional neural networks (BCNN) trained solely on binary weight values can achieve competitive rates of accuracy comparable to those of their analog weight counterparts. These findings highlight the dominance of the LRS for CSI RRAM with self‐compliance in a weighted configuration and minimal influence of the high resistance state despite substantial fluctuations for flexible CSI RRAM under varying bending radii. With its unique electrical switching capabilities, the CSI RRAM is highly anticipated to emerge as a promising candidate for embedded AI systems, especially in IoT devices and wearables.

show abstract

Section: Resultsmentioning

confidence: 99%

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Kumar,

Krishnaiah,

Park

et al. 2023

Adv Funct Materials

View full text Add to dashboard Cite

show abstract

“…The width multiplier and resolution multiplier were introduced into MobileNet to reduce the amount of calculation and the number of parameters and to construct a model with a smaller and lower computational cost. The accuracy of MobileNetV1 is slightly lower than that of VGG16 but better than that of GoogLeNet [245]. However, MobileNet has absolute advantages in terms of the calculation and parameter volume.…”

Section: Mobilenetmentioning

confidence: 91%

Deep Learning for Medical Image-Based Cancer Diagnosis

Jiang

Wang

et al. 2023

Cancers

View full text Add to dashboard Cite

(1) Background: The application of deep learning technology to realize cancer diagnosis based on medical images is one of the research hotspots in the field of artificial intelligence and computer vision. Due to the rapid development of deep learning methods, cancer diagnosis requires very high accuracy and timeliness as well as the inherent particularity and complexity of medical imaging. A comprehensive review of relevant studies is necessary to help readers better understand the current research status and ideas. (2) Methods: Five radiological images, including X-ray, ultrasound (US), computed tomography (CT), magnetic resonance imaging (MRI), positron emission computed tomography (PET), and histopathological images, are reviewed in this paper. The basic architecture of deep learning and classical pretrained models are comprehensively reviewed. In particular, advanced neural networks emerging in recent years, including transfer learning, ensemble learning (EL), graph neural network, and vision transformer (ViT), are introduced. Five overfitting prevention methods are summarized: batch normalization, dropout, weight initialization, and data augmentation. The application of deep learning technology in medical image-based cancer analysis is sorted out. (3) Results: Deep learning has achieved great success in medical image-based cancer diagnosis, showing good results in image classification, image reconstruction, image detection, image segmentation, image registration, and image synthesis. However, the lack of high-quality labeled datasets limits the role of deep learning and faces challenges in rare cancer diagnosis, multi-modal image fusion, model explainability, and generalization. (4) Conclusions: There is a need for more public standard databases for cancer. The pre-training model based on deep neural networks has the potential to be improved, and special attention should be paid to the research of multimodal data fusion and supervised paradigm. Technologies such as ViT, ensemble learning, and few-shot learning will bring surprises to cancer diagnosis based on medical images.

show abstract

“…It reviews the following four dimensions: (1) DNN model, (2) Hardware aspects, (3) Resources optimization, and (4) Application perspective. Finally, [20] explains the different DNN optimization techniques presented in recent research. It reviewed studies aiming at the implementation of DNN models on FPGAs.…”

Section: A Papers With Similar Backgroundmentioning

confidence: 99%

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Moghaddasi,

Gorgin,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Nowadays, artificial intelligence (AI) and deep learning (DL) progressively adapt to various spheres of our lives. These disciplines contain safety-critical applications such as autonomous driving with a high risk of human injury in the case of malfunction, requiring a high promise of dependability. Even the dependability becomes more crucial as shrinking CMOS technology feature size worsens the resilience concerns due to factors like aging. This paper addresses the overarching dependability issue of advanced deep neural networks (DNN) accelerators from the aging perspective. Especially, a comprehensive survey and taxonomy of techniques used to evaluate and mitigate aging effects are introduced. We cover different aging effects like permanent faults, timing errors, and lifetime issues. We review research by the layer-wise approach and categorize several resilience classes to bring out major features. The concluding part of this review highlights the questions answered and several future research directions. This study is expected to benefit researchers in different areas of DNN deployment, especially the dependability of this emergent paradigm.

show abstract

Accelerating Deep Neural Networks implementation: A survey

Cited by 22 publications

References 92 publications

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Deep Learning for Medical Image-Based Cancer Diagnosis

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Contact Info

Product

Resources

About

Accelerating Deep Neural Networks implementation: A survey

Cited by 22 publications

References 92 publications

Multibit, Lead‐Free Cs2SnI6 Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Multibit, Lead‐Free Cs2SnI6 Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Deep Learning for Medical Image-Based Cancer Diagnosis

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Contact Info

Product

Resources

About

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application

Multibit, Lead‐Free Cs₂SnI₆ Resistive Random Access Memory with Self‐Compliance for Improved Accuracy in Binary Neural Network Application