Large-Scale Multimodal Gesture Recognition Using Heterogeneous Networks

Wang, Huogen; Wang, Pichao; Song, Zhanjie; Li, Wanqing

doi:10.1109/iccvw.2017.370

Cited by 57 publications

(63 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, multi-modal approaches for improved emotion recognition were also discussed in both this work and Ref. 270. An end-to-end architecture incorporating temporal convolutions and bidirectional recurrence was proposed in Ref.…”

Section: Iris Recognitionmentioning

confidence: 99%

A Review on Human-Computer Interaction and Intelligent Robots

Ren

Bao

2020

Int. J. Info. Tech. Dec. Mak.

100

View full text Add to dashboard Cite

In the field of artificial intelligence, human–computer interaction (HCI) technology and its related intelligent robot technologies are essential and interesting contents of research. From the perspective of software algorithm and hardware system, these above-mentioned technologies study and try to build a natural HCI environment. The purpose of this research is to provide an overview of HCI and intelligent robots. This research highlights the existing technologies of listening, speaking, reading, writing, and other senses, which are widely used in human interaction. Based on these same technologies, this research introduces some intelligent robot systems and platforms. This paper also forecasts some vital challenges of researching HCI and intelligent robots. The authors hope that this work will help researchers in the field to acquire the necessary information and technologies to further conduct more advanced research.

show abstract

Section: Iris Recognitionmentioning

confidence: 99%

A Review on Human-Computer Interaction and Intelligent Robots

Ren

Bao

2020

Int. J. Info. Tech. Dec. Mak.

100

View full text Add to dashboard Cite

show abstract

“…The superior performance of our method demonstrates the effectiveness of our proposed method. Table 7 shows the performance of the two-Streams [8], 3D CNN [10], ConvLSTM [19], DI + CNN [19], and the proposed methods on the four categories of actions in the NTU RGB + D action dataset using depth modality alone and cross-subject protocol. As expected, the proposed method outperformed all other methods not only categories C3 and C4 but also the other two categories as well.…”

Section: Feature Fusionmentioning

confidence: 99%

“…Although the proposed method obtains outstanding performance overall, we observe that this method has relatively lower performance in actions such as "touch head", "sneeze/cough," "writing," and "eating a snack." Then a comparison between the proposed method and the popular approaches such as two-Streams [8], 3D CNN [10], ConvLSTM [19], DI+CNN [19] in these actions are made. Based on the comparison result, the proposed method achieves better performance than other approaches in these actions.…”

Section: Feature Fusionmentioning

confidence: 99%

“…The proposed hybrid network is evaluated and verified on the ChaLearn LAP IsoGD Dataset [13], NTU RGB+D Dataset [14] and Multi-modal, Multi-view and Interactive(M 2 I) Dataset [18]. This paper is an extension of the conference paper [19]. The extension includes WDI, feature level fusion using Canonical Correlation Analysis, a detailed justification of the proposed network, additional experiments on the NTU RGB+D Dataset and M 2 I Dataset and comparison with the methods reported recently.…”

Section: Introductionmentioning

confidence: 97%

See 1 more Smart Citation

A Hybrid Network for Large-Scale Action Recognition from RGB and Depth Modalities

Wang

Song

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

The paper presents a novel hybrid network for large-scale action recognition from multiple modalities. The network is built upon the proposed weighted dynamic images. It effectively leverages the strengths of the emerging Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) based approaches to specifically address the challenges that occur in large-scale action recognition and are not fully dealt with by the state-of-the-art methods. Specifically, the proposed hybrid network consists of a CNN based component and an RNN based component. Features extracted by the two components are fused through canonical correlation analysis and then fed to a linear Support Vector Machine (SVM) for classification. The proposed network achieved state-of-the-art results on the ChaLearn LAP IsoGD, NTU RGB+D and Multi-modal & Multi-view & Interactive ( M 2 I ) datasets and outperformed existing methods by a large margin (over 10 percentage points in some cases).

show abstract

“…At the submission time, our approach is in the second place as shown in Table 2. Zhang et al [32] RGBD + Flow 58.65 Wang et al [27] RGBD + Flow 60.81 ResC3D [16] RGBD + Flow 64.40 Table 5: Comparison with state-of-the-art on ChaLearn dataset in validation accuracy.…”

Section: C3dmentioning

confidence: 99%

Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition

Köpüklü

Köse

Rigoll

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

103

View full text Add to dashboard Cite

Acquiring spatio-temporal states of an action is the most crucial step for action classification. In this paper, we propose a data level fusion strategy, Motion Fused Frames (MFFs), designed to fuse motion information into static images as better representatives of spatio-temporal states of an action. MFFs can be used as input to any deep learning architecture with very little modification on the network. We evaluate MFFs on hand gesture recognition tasks using three video datasets-Jester, ChaLearn LAP IsoGD and NVIDIA Dynamic Hand Gesture Datasets-which require capturing long-term temporal relations of hand movements. Our approach obtains very competitive performance on Jester and ChaLearn benchmarks with the classification accuracies of 96.28% and 57.4%, respectively, while achieving state-of-theart performance with 84.7% accuracy on NVIDIA benchmark.

show abstract

Large-Scale Multimodal Gesture Recognition Using Heterogeneous Networks

Cited by 57 publications

References 27 publications

A Review on Human-Computer Interaction and Intelligent Robots

A Review on Human-Computer Interaction and Intelligent Robots

A Hybrid Network for Large-Scale Action Recognition from RGB and Depth Modalities

Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition

Contact Info

Product

Resources

About