FunnyNet: Audiovisual Learning of Funny Moments in Videos

Liu, Zhisong; Courant, Robin; Kalogeiton, Vicky

doi:10.1007/978-3-031-26316-3_26

Cited by 3 publications

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Proxemics-net++: classification of human interactions in still images

Jiménez-Velasco,

Zafra-Palma,

Muñoz-Salinas

et al. 2024

Pattern Anal Applic

View full text Add to dashboard Cite

Human interaction recognition (HIR) is a significant challenge in computer vision that focuses on identifying human interactions in images and videos. HIR presents a great complexity due to factors such as pose diversity, varying scene conditions, or the presence of multiple individuals. Recent research has explored different approaches to address it, with an increasing emphasis on human pose estimation. In this work, we propose Proxemics-Net++, an extension of the Proxemics-Net model, capable of addressing the problem of recognizing human interactions in images through two different tasks: the identification of the types of “touch codes” or proxemics and the identification of the type of social relationship between pairs. To achieve this, we use RGB and body pose information together with the state-of-the-art deep learning architecture, ConvNeXt, as the backbone. We performed an ablative analysis to understand how the combination of RGB and body pose information affects these two tasks. Experimental results show that body pose information contributes significantly to proxemic recognition (first task) as it allows to improve the existing state of the art, while its contribution in the classification of social relations (second task) is limited due to the ambiguity of labelling in this problem, resulting in RGB information being more influential in this task.

show abstract

Proxemics-net++: classification of human interactions in still images

Jiménez-Velasco,

Zafra-Palma,

Muñoz-Salinas

et al. 2024

Pattern Anal Applic

View full text Add to dashboard Cite

show abstract

Name your style: text-guided artistic style transfer

Liu

Wang

Siu

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Image style transfer has attracted widespread attention in the past years. Despite its remarkable results, it requires additional style images available as references, making it less flexible and inconvenient. Using text is the most natural way to describe the style. Text can describe implicit abstract styles, like styles of specific artists or art movements. In this work, we propose a text-driven style transfer (TxST) that leverages advanced image-text encoders to control arbitrary style transfer. We introduce a contrastive training strategy to effectively extract style descriptions from the image-text model (i.e., CLIP), which aligns stylization with the text description. To this end, we also propose a novel cross-attention module to fuse style and content features. Finally, we achieve an arbitrary artist-aware style transfer to learn and transfer specific artistic characters such as Picasso, oil painting, or a rough sketch. Extensive experiments demonstrate that our approach outperforms the stateof-the-art methods. Moreover, it can mimic the styles of one or many artists to achieve attractive results, thus highlighting a promising future direction.

show abstract