Research on Transfer Learning of Vision-based Gesture Recognition

Wu, Bixiao; Yang, Chenguang; Zhong, Junpei

doi:10.1007/s11633-020-1273-9

Cited by 20 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These devices are widely utilized in HRC and robot teleoperation tasks. [45,51,52,93,94] Beyond the cameras noted, radar, laser, and even mirrors can serve to capture visual information. Zhang et al proposed ReflectU, which is based on mirror reflection, to detect human motion in human and multirobot interaction.…”

Section: Vision-based Interfacementioning

confidence: 99%

See 1 more Smart Citation

Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A Survey

Wang,

Zheng,

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Human–robot interaction (HRI) has escalated in notability in recent years, and multimodal communication and control strategies are necessitated to guarantee a secure, efficient, and intelligent HRI experience. In spite of the considerable focus on multimodal HRI, comprehensive disquisitions delineating various modalities and intricately analyzing their combinations remain elusive, consequently limiting holistic understanding and future advancements. This article aspires to bridge this inadequacy by conducting a profound exploration of multimodal HRI, predominantly concentrating on four principal modalities: vision, auditory and language, haptics, and physiological sensing. An extensive review encapsulating algorithmic dissection, interface devices, and applicative dimensions forms part of this discourse. This manuscript distinctively combines multimodal HRI with cognitive science, deeply probing into the three dimensions, perception, cognition, and action, thereby demystifying algorithms intrinsic to multimodal HRI. Finally, it accentuates the empirical challenges and contours preemptive trajectories for multimodal HRI in human‐centric smart manufacturing.

show abstract

Section: Vision-based Interfacementioning

confidence: 99%

“…[42][43][44] In contrast, hand gesture recognition finds utility in scenarios such as collaborative manipulation, robot programming, surgical robot teleoperation, and beyond. [45][46][47][48][49][50][51][52] Table 1. Vision-based technologies.…”

Section: Human Posementioning

confidence: 99%

Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A Survey

Wang,

Zheng,

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…This may lead to poor generalization performance and model failure in real-world applications. In order to address this problem, domain adaptation techniques can be used to improve the generalization performance of the visual recognition model on the target dataset [ 21 , 22 , 23 , 24 ]. These techniques aim to align the statistical properties of the source and target domains by reducing the distribution shift between them.…”

Section: Introductionmentioning

confidence: 99%

Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition

Baptista

Santos

Silva

et al. 2023

Sensors

View full text Add to dashboard Cite

Hand gesture recognition from images is a critical task with various real-world applications, particularly in the field of human–robot interaction. Industrial environments, where non-verbal communication is preferred, are significant areas of application for gesture recognition. However, these environments are often unstructured and noisy, with complex and dynamic backgrounds, making accurate hand segmentation a challenging task. Currently, most solutions employ heavy preprocessing to segment the hand, followed by the application of deep learning models to classify the gestures. To address this challenge and develop a more robust and generalizable classification model, we propose a new form of domain adaptation using multi-loss training and contrastive learning. Our approach is particularly relevant in industrial collaborative scenarios, where hand segmentation is difficult and context-dependent. In this paper, we present an innovative solution that further challenges the existing approach by testing the model on an entirely unrelated dataset with different users. We use a dataset for training and validation and demonstrate that contrastive learning techniques in simultaneous multi-loss functions provide superior performance in hand gesture recognition compared to conventional approaches in similar conditions.

show abstract

“…Considering larger datasets, fully supervised learning of CNNs was possible without over fitting additional measure of parameters (12) . Recent studies revealed that, the restrictions of CNN that was trained for large datasets, for instance, ILSVRC shall be used in object recognition or image classification tasks when the information was restricted, resulting in better execution of usual representations (13)(14)(15) . Very few research works were done on sentimental analysis methods dependent on CNN for visual sentimental predictions.…”

Section: Introductionmentioning

confidence: 99%

A Novel Fusion Attention Algorithm for Sentimental Image Analysis

Roseline¹,

Chellam²

2022

IJST

View full text Add to dashboard Cite

Objectives: To implement a novel and hybrid methodology for finding out the positive features when using convolutional neural networks (CNNs) for visual sentiment analysis. To achieve increased accuracy, precision and recall by using this proposed fusion attention methodology. Methods: This study proposes a modified methodology encompassing spatial attention, channel attention as well as squeeze excitation modules. An enhanced approach on the basis of convolutional neural networks was used here which utilizes convolution operators by combining both spatial and channel-based data. Moreover, we have incorporated three considerations like spatial, channel as well as squeeze and excitation at various levels for attaining optimal results. Findings: The accuracy of the existing approaches was 59.88%, 60.06%, 59.28% and 62.89%, but the proposed fusion attention method showed increased accuracy of 64.15%. Similarly, the F1 score of existing approaches are 0.464804, 0.250164, 0.474129 and 0.2574, but the proposed method revealed increased F1 score of 0.512933. Furthermore, the proposed algorithm showed precision and recall of 0.560896 and 0.472526 which were better when compared with the existing approaches like Res-Target, Resnet50, Alexnet and VGG16. Novelty: The novel feature of this proposed fusion attention algorithm was that it incorporates a hybrid approach in which the image together with convolution passes through channel attention, spatial attention as well as squeeze and excitation so as to attain increased accuracy, but most of the existing approaches have used only channel attention and spatial attention modules. In this proposed method, the algorithm performs convolution in 64-bit, 128-bit and 256-bit respectively together in which the three attentions were interchanged in each convolution, which were not prevalent in the existing approaches.

show abstract

Research on Transfer Learning of Vision-based Gesture Recognition

Cited by 20 publications

References 31 publications

Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A Survey

Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A Survey

Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition

A Novel Fusion Attention Algorithm for Sentimental Image Analysis

Contact Info

Product

Resources

About