Design and Performance Evaluation of Multispectral Sensing Algorithms on CPU, GPU, and FPGA

Menon, Vivek; Siddiqui, Saquib Ahmad; Rao, S.; Schmidt, Andrew G.; French, Matthew; Chirayath, Ved; Li, Alan

doi:10.1109/aero50100.2021.9438307

“…However, cameras can lead to privacy concerns for users, especially in residential environments, which limits their applications. Gesture recognition has become a common interaction pattern in recent years with LiDAR, radar, infrared, resistive, and capacitive sensors [5]- [7], [14]- [18]. CapBand [7] is developed to recognize static hand gestures using capacitive sensors on a wrist band with a convolutional neural network (CNN).…”

Section: A Sensors and Pattern Recognitionmentioning

confidence: 99%

An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

Liu

¹

,

Panahi

²

,

Andrews

³

et al. 2022

View full text Add to dashboard Cite

Upper-Extremity motor impairment affects millions of Americans due to cerebrovascular incidents, spinal cord injuries, or brain trauma. Current therapy practices used to assist these individuals in regaining motor functionality often require extensive time at rehabilitation facilities with potentially prohibitive travel or financial costs. This work presents a mobile low-cost field programmable gate array (FPGA)-smart rehabilitation system that can be used in home environments. The prototype is a rehabilitation table instrumented with a capacitive sensor array (CSA) to track upper-extremity motions of the user through proximity or touch. In addition, inertial measurement units (IMUs) are placed on the affected upper limb and combined with the CSA data with our sensor fusion signal processing architecture. Motions are classified and evaluated using multi-task convolutional recurrent neural networks with three additional motion quality output classes to personalize recognition based on the particular motor skills of each patient. The prototype achieves above 99% accuracy with 32-bit fixed-point format implementation for recognizing dynamic motions and identifying unnatural characteristics (i.e., tremor or limited flexion and extension) in upper limb motions based on sensor values. The convolutional recurrent neural network (C-RNN) fusion classification network is implemented on a 200 MHz Zynq ZCU104 FPGA using an HLS-based design optimized with pipelining and parallelism techniques and achieves 5.4x speedup compared to ARM ® Cortex-A53 implementation running at an operating frequency of 1.3 GHz. The prototype is also demonstrated to perform the machine learning classification in real-time.

show abstract

An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

Liu

¹

,

Panahi

²

,

Andrews

³

et al. 2020

2020 International Conference on Field-Programmable Technology (ICFPT)

View full text Add to dashboard Cite

Upper-Extremity motor impairment affects millions of Americans due to cerebrovascular incidents, spinal cord injuries, or brain trauma. Current therapy practices used to assist these individuals in regaining motor functionality often require extensive time at rehabilitation facilities with potentially prohibitive travel or financial costs. This work presents a mobile low-cost field programmable gate array (FPGA)-smart rehabilitation system that can be used in home environments. The prototype is a rehabilitation table instrumented with a capacitive sensor array (CSA) to track upper-extremity motions of the user through proximity or touch. In addition, inertial measurement units (IMUs) are placed on the affected upper limb and combined with the CSA data with our sensor fusion signal processing architecture. Motions are classified and evaluated using multi-task convolutional recurrent neural networks with three additional motion quality output classes to personalize recognition based on the particular motor skills of each patient. The prototype achieves above 99% accuracy with 32-bit fixed-point format implementation for recognizing dynamic motions and identifying unnatural characteristics (i.e., tremor or limited flexion and extension) in upper limb motions based on sensor values. The convolutional recurrent neural network (C-RNN) fusion classification network is implemented on a 200 MHz Zynq ZCU104 FPGA using an HLS-based design optimized with pipelining and parallelism techniques and achieves 5.4x speedup compared to ARM ® Cortex-A53 implementation running at an operating frequency of 1.3 GHz. The prototype is also demonstrated to perform the machine learning classification in real-time.

show abstract

HLS Taking Flight: Toward Using High-Level Synthesis Techniques in a Space-Borne Instrument

Sudvarg,

Zhao,

Htet

et al. 2024

Proceedings of the 21st ACM International Conference on Computing Frontiers

1

0

View full text Add to dashboard Cite

Design and Performance Evaluation of Multispectral Sensing Algorithms on CPU, GPU, and FPGA

Cited by 3 publications

References 13 publications

An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

HLS Taking Flight: Toward Using High-Level Synthesis Techniques in a Space-Borne Instrument

Contact Info

Product

Resources

About