Multi-stream pose convolutional neural networks for human interaction recognition in images

Tanisik, Gokhan; Zalluhoglu, Cemil; Ikizler-Cinbis, Nazli

doi:10.1016/j.image.2021.116265

Cited by 5 publications

(1 citation statement)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They have attained 96.8% accuracy on HII data using the fine-tuned ResNet101 model. In another study, Tanisik et al [9] delved into the significance of human poses in discerning human interactions within still images. Their novel approach introduces a multi-stream convolutional neural network architecture, harmonizing diverse human pose information to enhance human interaction recognition.…”

Section: Related Workmentioning

confidence: 99%

Recognition of Human Interactions in Still Images using AdaptiveDRNet with Multi-level Attention

Dey,

Biswas,

2023

IJACSA

View full text Add to dashboard Cite

Human-Human Interaction Recognition (H2HIR) is a multidisciplinary field that combines computer vision, deep learning, and psychology. Its primary objective is to decode and understand the intricacies of human-human interactions. H2HIR holds significant importance across various domains as it enables machines to perceive, comprehend, and respond to human social behaviors, gestures, and communication patterns. This study aims to identify human-human interactions from just one frame, i.e. from an image. Diverging from the realm of video-based interaction recognition, a well-established research domain that relies on the utilization of spatio-temporal information, the complexity of the task escalates significantly when dealing with still images due to the absence of these intrinsic spatio-temporal features. This research introduces a novel deep learning model called AdaptiveDRNet with Multi-level Attention to recognize Human-Human (H2H) interactions. Our proposed method demonstrates outstanding performance on the Human-Human Interaction Image dataset (H2HID), encompassing 4049 meticulously curated images representing fifteen distinct human interactions and on the publicly accessible HII and HIIv2 related benchmark datasets. Notably, our proposed model excels with a validation accuracy of 97.20% in the classification of human-human interaction images, surpassing the performance of EfficientNet, InceptionResNetV2, NASNet Mobile, ConvXNet, ResNet50, and VGG-16 models. H2H interaction recognition's significance lies in its capacity to enhance communication, improve decision-making, and ultimately contribute to the well-being and efficiency of individuals and society as a whole.

show abstract

Section: Related Workmentioning

confidence: 99%

Recognition of Human Interactions in Still Images using AdaptiveDRNet with Multi-level Attention

Dey,

Biswas,

2023

IJACSA

View full text Add to dashboard Cite

show abstract

Proxemics-net++: classification of human interactions in still images

Jiménez-Velasco,

Zafra-Palma,

Muñoz-Salinas

et al. 2024

Pattern Anal Applic

View full text Add to dashboard Cite

Human interaction recognition (HIR) is a significant challenge in computer vision that focuses on identifying human interactions in images and videos. HIR presents a great complexity due to factors such as pose diversity, varying scene conditions, or the presence of multiple individuals. Recent research has explored different approaches to address it, with an increasing emphasis on human pose estimation. In this work, we propose Proxemics-Net++, an extension of the Proxemics-Net model, capable of addressing the problem of recognizing human interactions in images through two different tasks: the identification of the types of “touch codes” or proxemics and the identification of the type of social relationship between pairs. To achieve this, we use RGB and body pose information together with the state-of-the-art deep learning architecture, ConvNeXt, as the backbone. We performed an ablative analysis to understand how the combination of RGB and body pose information affects these two tasks. Experimental results show that body pose information contributes significantly to proxemic recognition (first task) as it allows to improve the existing state of the art, while its contribution in the classification of social relations (second task) is limited due to the ambiguity of labelling in this problem, resulting in RGB information being more influential in this task.

show abstract

Multi-level channel attention excitation network for human action recognition in videos

2023

Signal Processing: Image Communication

View full text Add to dashboard Cite

Multi-stream pose convolutional neural networks for human interaction recognition in images

Cited by 5 publications

References 49 publications

Recognition of Human Interactions in Still Images using AdaptiveDRNet with Multi-level Attention

Recognition of Human Interactions in Still Images using AdaptiveDRNet with Multi-level Attention

Proxemics-net++: classification of human interactions in still images

Multi-level channel attention excitation network for human action recognition in videos

Contact Info

Product

Resources

About