Wearable on-device deep learning system for hand gesture recognition based on FPGA accelerator

Jiang, Wenming; Ye, Xuelin; Chen, Ruiqi; Su, Feng; Lin, Meng-Ru; Ma, Yuhanxiao; Zhu, Yanxiang; Huang, Shizhen

doi:10.3934/mbe.2021007

Cited by 20 publications

(9 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, Convolutional Neural Network (CNN) and Multilayer Perceptron Neural Network (NN) are used in the recognition model to extract features and classify gestures, which helps achieve a recognition accuracy of up to 97%. Finally, Jiang W provides a software-hardware co-design method, which is worthy of reference for the design of edge devices in other scenarios [ 4 ]. With the rise of robotic surgical systems, medical patients using AI human-computer interaction technology and now requirements around electronic health records (EHR) are examine these systems as part of their larger sociotechnical systems.…”

Section: Related Workmentioning

confidence: 99%

Role of Human-Computer Interaction Healthcare System in the Teaching of Physiology and Medicine

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

With the increasingly severe aging of the population, the difficult and expensive medical treatment problems are becoming more and more prominent; the salary level of domestic doctors is not high, but the cost of training doctors is high, coupled with doctors’ work pressure and mental pressure; the number of candidates for medical school is decreasing year by year; medical talent is rare; and the allocation of medical staff is scarce. Health care is the basic guarantee for people’s good life, and the shortage of medical staff will have many impacts on health care. Human-computer interaction (HCI) is the study of people, computers, and their interaction. HCI refers to the communication between the user and the computer system, which is the two-way information exchange of various symbols and actions between the human and the computer. The purpose of this paper is to study a healthcare system with human-computer interaction through the client, apply the system to the teaching of physiology and medicine, and analyze its effects and functions in combination with various evaluation indicators. This paper selects teaching content, ease of use of human-computer interaction design, technical services, and user subjective satisfaction as evaluation indicators, and constructs an evaluation model for this. And it builds the physiology and medicine teaching system framework and healthcare system, and conducts tests and statistics on the teaching system. This paper combines online questionnaires, in-app survey feedback, and field visits to collect feedback from users and administrators. The final data show that the teaching system meets the requirements in four evaluation indicators: teaching content, ease of use of human-computer interaction design, technical services, and user subjective satisfaction. User satisfaction with these four aspects reached 86.33%, 95.17%, 63.83%, and 81.87%, respectively. It shows that the system is more popular and can meet the needs of most users.

show abstract

Section: Related Workmentioning

confidence: 99%

Role of Human-Computer Interaction Healthcare System in the Teaching of Physiology and Medicine

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Coffen et al constructed a multilayer long short-term memory model to analyse data collected from a finger-worn ring profile device and reached an accuracy from 75 to 95% per finger stroke, but their attempt to transform the model to a compressed TF Lite format to run on-device did not succeed [5]. Jiang et al proposed a wearable deep learning system capable of processing data on the end device locally [21]. Moreover, air-writing is more complicated than finger stroke recognition and involves more subtle distinctions.…”

Section: Related Workmentioning

confidence: 99%

A Wearable Real-Time Character Recognition System Based on Edge Computing-Enabled Deep Learning for Air-Writing

et al. 2022

View full text Add to dashboard Cite

Nowadays, wearable sensors play a vital role in the detection of human motions, innovating an alternate and intuitive form in human–computer interaction (HCI). In this study, we present a novel real-time wearable system for finger air-writing recognition in three-dimensional (3D) space based on the Arduino Nano 33 BLE Sense as an edge device, which can run TensorFlow Lite to realize recognition and classification on the device. This system enables users to have the freedom and flexibility to write characters (10 digits and 26 English lower-case letters) in free space by moving fingers and uses a deep learning algorithm to recognize 36 characters from the motion data captured by inertial measurement units (IMUs) and processed by a microcontroller, which are both embedded in an Arduino Nano 33 BLE Sense. We prepared 63000 air-writing stroke data samples of 35 subjects containing 18 males and 17 females for convolutional neural network (CNN) training and achieved a high recognition accuracy at 97.95%.

show abstract

“…Wearable fitness trackers, for example, are devices that track and monitor various physical activity metrics, such as steps taken, distance traveled and calories burned (Nuss et al , 2021). These devices can provide real-time feedback to users, helping them to monitor their progress and set goals for their movement training (Chow and Yang, 2020; Jiang et al , 2021). Other technologies that are used to assist movement training include video games that incorporate physical activity, such as dance or sports simulation games, as well as online training programs and apps that provide structured workouts and guidance Song et al (2019).…”

Section: Introductionmentioning

confidence: 99%

Immersive innovations: an examination of the efficacy and evolution of virtual reality in human movement training

Deng,

Wang,

Dong

et al. 2023

RIA

View full text Add to dashboard Cite

Purpose This systematic review, following preferred reporting items for systematic reviews and meta-analysis guidelines, rigorously investigates the emergent role of virtual reality (VR) technology in human movement training. The purpose of this study is to explore the effectiveness and evolution of VR in enhancing movement training experiences. Design/methodology/approach Acknowledging its pivotal role in diverse applications, such as sports and rehabilitation, human movement training is currently experiencing accelerated evolution, facilitated by the proliferation of wearable devices and mobile applications. This review conducted an exhaustive search across five different electronic databases, such as Web of Science, PubMed and ProQuest, resulting in the selection of 69 eligible articles published within the past five years. It also integrates 40 studies into a narrative summary, categorized based on the level of immersion offered by respective VR systems. Findings Enhanced immersion in VR potentially augments the effectiveness of movement training by engendering more realistic and captivating experiences for users. The immersive and interactive environments provided by VR technology enable tailored training experiences accompanied by precise, objective feedback. This review highlights the benefits of VR in human movement training and its potential to revolutionize the way training is conducted. Originality/value This systematic review contributes significantly to the existing literature by providing a comprehensive examination of the efficacy and evolution of VR in human movement training. By organizing the findings based on the level of immersion offered by VR systems, it provides valuable insights into the importance of immersion in enhancing training outcomes. In addition, this study identifies the need for future research focusing on the impacts of VR on learning and performance, as well as strategies to optimize its effectiveness and improve accessibility.

show abstract

Wearable on-device deep learning system for hand gesture recognition based on FPGA accelerator

Cited by 20 publications

References 45 publications

Role of Human-Computer Interaction Healthcare System in the Teaching of Physiology and Medicine

Role of Human-Computer Interaction Healthcare System in the Teaching of Physiology and Medicine

A Wearable Real-Time Character Recognition System Based on Edge Computing-Enabled Deep Learning for Air-Writing

Immersive innovations: an examination of the efficacy and evolution of virtual reality in human movement training

Contact Info

Product

Resources

About