Real-Time Gesture Recognition Using 3D Sensory Data and a Light Convolutional Neural Network

Diliberti, Nicholas; Peng, Chao; Kaufman, Christopher L.; Dong, Yangzi; Hansberger, Jeffrey T.

doi:10.1145/3343031.3350958

Cited by 16 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 252 Accuracy (98.6%) Hand Gesture Recognition (HAR) system for Human-Computer Interaction (HCI) based on time-dependent data from IMU sensors. [ 183 ] Motion capturing gloves are designed using 3D sensory data Classification model with ANN. 6700 Accuracy (98%) Data gloves with IMU sensors are used to capture finger and palm movements.…”

Section: Human Activity Detection Using Deep Learning Techniquesmentioning

confidence: 99%

“…Data gloves collect information and transmit it wirelessly and in real-time to a compatible receiving device. With 0.1 s of predicting, the system reached a speed of 8.94 milliseconds per frame and an accuracy of 98% [ 183 ]. Using this data glove system, a multi-sensor motion capture system capable of identifying six motions was constructed.…”

Section: Human Activity Detection Using Deep Learning Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Review of Wearable Devices and Data Collection Considerations for Connected Health

Vijayan

Connolly

Condell³

et al. 2021

Sensors

205

View full text Add to dashboard Cite

Wearable sensor technology has gradually extended its usability into a wide range of well-known applications. Wearable sensors can typically assess and quantify the wearer’s physiology and are commonly employed for human activity detection and quantified self-assessment. Wearable sensors are increasingly utilised to monitor patient health, rapidly assist with disease diagnosis, and help predict and often improve patient outcomes. Clinicians use various self-report questionnaires and well-known tests to report patient symptoms and assess their functional ability. These assessments are time consuming and costly and depend on subjective patient recall. Moreover, measurements may not accurately demonstrate the patient’s functional ability whilst at home. Wearable sensors can be used to detect and quantify specific movements in different applications. The volume of data collected by wearable sensors during long-term assessment of ambulatory movement can become immense in tuple size. This paper discusses current techniques used to track and record various human body movements, as well as techniques used to measure activity and sleep from long-term data collected by wearable technology devices.

show abstract

Section: Human Activity Detection Using Deep Learning Techniquesmentioning

confidence: 99%

Section: Human Activity Detection Using Deep Learning Techniquesmentioning

confidence: 99%

Review of Wearable Devices and Data Collection Considerations for Connected Health

Vijayan

Connolly

Condell³

et al. 2021

Sensors

205

View full text Add to dashboard Cite

show abstract

“…Embodied conversational agents could become a part of people's everyday life, but to date they are not fully capable of understanding and realizing the seemingly "natural" and socialized aspects of human gesture recognition by vision and hand tracking [54][55][56][57][58][59] and speech recognition and analysis [60,61].…”

Section: Technology: Implementation Of Affective Smart Agent Avatars In Vrmentioning

confidence: 99%

Identifying research gaps: A review of virtual patient education and self-management

Brucker-Kley

Kleinberger

Keller

et al. 2021

THC

View full text Add to dashboard Cite

BACKGROUND: Avatars in Virtual Reality (VR) can not only represent humans, but also embody intelligent software agents that communicate with humans, thus enabling a new paradigm of human-machine interaction. OBJECTIVE: The research agenda proposed in this paper by an interdisciplinary team is motivated by the premise that a conversation with a smart agent avatar in VR means more than giving a face and body to a chatbot. Using the concrete communication task of patient education, this research agenda is rather intended to explore which patterns and practices must be constructed visually, verbally, para- and nonverbally between humans and embodied machines in a counselling context so that humans can integrate counselling by an embodied VR smart agent into their thinking and acting in one way or another. METHODS: The scientific literature in different bibliographical databases was reviewed. A qualitative narrative approach was applied for analysis. RESULTS: A research agenda is proposed which investigates how recurring consultations of patients with healthcare professionals are currently conducted and how they could be conducted with an embodied smart agent in immersive VR. CONCLUSIONS: Interdisciplinary teams consisting of linguists, computer scientists, visual designers and health care professionals are required which need to go beyond a technology-centric solution design approach. Linguists’ insights from discourse analysis drive the explorative experiments to identify test and discover what capabilities and attributes the smart agent in VR must have, in order to communicate effectively with a human being.

show abstract

“…Although camera-based approaches have the advantage of being able to utilize a variety of information captured by the camera, they often suffer from problems of self-occlusion and a limited camera workspace. On the other hand, sensor-based methods generally rely on 3D kinematic information, such as angular and linear parameters, measured from wearable or handheld devices [12][13][14][15][16][17]. The major advantage of these methods is that 3D kinematic information can be obtained continuously without suffering from the self-occlusion problem.…”

Section: Related Workmentioning

confidence: 99%

Gravity Control-Based Data Augmentation Technique for Improving VR User Activity Recognition

2021

View full text Add to dashboard Cite

The neural-network-based human activity recognition (HAR) technique is being increasingly used for activity recognition in virtual reality (VR) users. The major issue of a such technique is the collection large-scale training datasets which are key for deriving a robust recognition model. However, collecting large-scale data is a costly and time-consuming process. Furthermore, increasing the number of activities to be classified will require a much larger number of training datasets. Since training the model with a sparse dataset can only provide limited features to recognition models, it can cause problems such as overfitting and suboptimal results. In this paper, we present a data augmentation technique named gravity control-based augmentation (GCDA) to alleviate the sparse data problem by generating new training data based on the existing data. The benefits of the symmetrical structure of the data are that it increased the number of data while preserving the properties of the data. The core concept of GCDA is two-fold: (1) decomposing the acceleration data obtained from the inertial measurement unit (IMU) into zero-gravity acceleration and gravitational acceleration, and augmenting them separately, and (2) exploiting gravity as a directional feature and controlling it to augment training datasets. Through the comparative evaluations, we validated that the application of GCDA to training datasets showed a larger improvement in classification accuracy (96.39%) compared to the typical data augmentation methods (92.29%) applied and those that did not apply the augmentation method (85.21%).

show abstract

Real-Time Gesture Recognition Using 3D Sensory Data and a Light Convolutional Neural Network

Cited by 16 publications

References 26 publications

Review of Wearable Devices and Data Collection Considerations for Connected Health

Review of Wearable Devices and Data Collection Considerations for Connected Health

Identifying research gaps: A review of virtual patient education and self-management

Gravity Control-Based Data Augmentation Technique for Improving VR User Activity Recognition

Contact Info

Product

Resources

About