Efficient Inference Of Image-Based Neural Network Models In Reconfigurable Systems With Pruning And Quantization

Flich, Jose; Medina, Laura; Catalan, Izan; Hernández, Carles; Bragagnolo, Andrea; Auzanneau, Fabrice

doi:10.1109/icip46576.2022.9897752

Cited by 3 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inference time has been measured from the host. To carry out the tests we used the HLSinf [20] accelerator configured to use FP32 data type. Although it is advisable to use fixed point data types in FPGAs, the accelerator achieves better performance with FP32.…”

Section: Design Evaluationmentioning

confidence: 99%

Towards Efficient Neural Network Model Parallelism on Multi-FPGA Platforms

Agut¹,

Tornero²,

Flich³

2023

2023 Design, Automation &Amp; Test in Europe Conference &Amp; Exhibition (DATE)

View full text Add to dashboard Cite

Nowadays, convolutional neural networks (CNN) are common in a wide range of applications. Their high accuracy and efficiency contrast with their computing requirements, leading to the search for efficient hardware platforms. FPGAs are suitable due to their flexibility, energy efficiency and low latency. However, the ever increasing complexity of CNNs demands higher capacity devices, forcing the need for multi-FPGA platforms. In this paper, we present a multi-FPGA platform with distributed shared memory support for the inference of CNNs. Our solution, in contrast with previous works, enables combining different model parallelism strategies applied to CNNs, thanks to the distributed shared memory support. For a four FPGA setting, the platform reduces the execution time of 2D convolutions by a factor of 3.95 when compared to single FPGA. The inference of standard CNN models is improved by factors ranging 3.63-3.87.

show abstract

Section: Design Evaluationmentioning

confidence: 99%

Towards Efficient Neural Network Model Parallelism on Multi-FPGA Platforms

Agut¹,

Tornero²,

Flich³

2023

2023 Design, Automation &Amp; Test in Europe Conference &Amp; Exhibition (DATE)

View full text Add to dashboard Cite

show abstract

“…The AI hardware accelerator, AI-Inference library, and an acceleration runtime method created in this work comprise the SELENE Accelerator Framework (SAF). It works as follows: first, the European Distributed Deep Learning (EDDL) [23] inference library initializes the HLSInf [24] HW accelerator using the generated JSON configuration file. Next, the inference input data (i.e., the data to be processed) is loaded in the main memory shared with the accelerator.…”

mentioning

confidence: 99%

HPC Platform for Railway Safety-Critical Functionalities Based on Artificial Intelligence

et al. 2023

Self Cite

View full text Add to dashboard Cite

The automation of railroad operations is a rapidly growing industry. In 2023, a new European standard for the automated Grade of Automation (GoA) 2 over European Train Control System (ETCS) driving is anticipated. Meanwhile, railway stakeholders are already planning their research initiatives for driverless and unattended autonomous driving systems. As a result, the industry is particularly active in research regarding perception technologies based on Computer Vision (CV) and Artificial Intelligence (AI), with outstanding results at the application level. However, executing high-performance and safety-critical applications on embedded systems and in real-time is a challenge. There are not many commercially available solutions, since High-Performance Computing (HPC) platforms are typically seen as being beyond the business of safety-critical systems. This work proposes a novel safety-critical and high-performance computing platform for CV- and AI-enhanced technology execution used for automatic accurate stopping and safe passenger transfer railway functionalities. The resulting computing platform is compatible with the majority of widely-used AI inference methodologies, AI model architectures, and AI model formats thanks to its design, which enables process separation, redundant execution, and HW acceleration in a transparent manner. The proposed technology increases the portability of railway applications into embedded systems, isolates crucial operations, and effectively and securely maintains system resources.

show abstract

Exploiting neural networks bit-level redundancy to mitigate the impact of faults at inference

Catalán,

Flich,

Hernández

2024

J Supercomput

View full text Add to dashboard Cite

Neural networks are widely used in critical environments such as healthcare, autonomous vehicles, or video surveillance. To ensure the safety of the systems that rely on their functionality, it is essential to validate their correct behaviour in the presence of faults. This paper studies the behaviour of state-of-the-art neural network models with fault injection in their weights. For this purpose, we analyse the sensitivity of these models and identify the impact of bit flips on their accuracy. To mitigate the effects of faults, we introduce two mechanisms that leverage bit-level redundancy for protection. The first mechanism, Fixed Protection, safeguards consecutive sets of bits, while the second, Variable Protection, targets non-consecutive bits. Our findings demonstrate that, on average, random bit flip faults cause the accuracy of the original models to drop by 1.3% to over 3%. However, with our protection mechanisms in place, accuracy reductions are significantly minimised, ranging from only 0.0001% to 0.4%.

show abstract

Efficient Inference Of Image-Based Neural Network Models In Reconfigurable Systems With Pruning And Quantization

Cited by 3 publications

References 17 publications

Towards Efficient Neural Network Model Parallelism on Multi-FPGA Platforms

Towards Efficient Neural Network Model Parallelism on Multi-FPGA Platforms

HPC Platform for Railway Safety-Critical Functionalities Based on Artificial Intelligence

Exploiting neural networks bit-level redundancy to mitigate the impact of faults at inference

Contact Info

Product

Resources

About