Large Scale Holistic Video Understanding

Diba, Ali; Fayyaz, Mohsen; Sharma, Vivek; Paluri, Manohar; Gall, Jürgen; Stiefelhagen, Rainer; Gool, Luc Van

doi:10.1007/978-3-030-58558-7_35

Cited by 86 publications

(83 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MTL has been applied to other problems too. This includes various domains, such as language [24], [98], [99], audio [100], video [101], [102], and robotics [103], [104], as well as with different learning paradigms, such as reinforcement learning [105], [106], self-supervised learning [107], semisupervised learning [108], [109] and active learning [110], [111]. Surprisingly, in the deep learning era, very few works have considered MTL under the semi-supervised or active learning setting.…”

Section: Othermentioning

confidence: 99%

Multi-Task Learning for Dense Prediction Tasks: A Survey

Vandenhende¹,

Georgoulis²,

Gansbeke³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

428

222

View full text Add to dashboard Cite

With the advent of deep learning, many dense prediction tasks, i.e. tasks that produce pixel-level predictions, have seen significant performance improvements. The typical approach is to learn these tasks in isolation, that is, a separate neural network is trained for each individual task. Yet, recent multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint, by jointly tackling multiple tasks through a learned shared representation. In this survey, we provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision, explicitly emphasizing on dense prediction tasks. Our contributions concern the following. First, we consider MTL from a network architecture point-of-view. We include an extensive overview and discuss the advantages/disadvantages of recent popular MTL models. Second, we examine various optimization methods to tackle the joint learning of multiple tasks. We summarize the qualitative elements of these works and explore their commonalities and differences. Finally, we provide an extensive experimental evaluation across a variety of dense prediction benchmarks to examine the pros and cons of the different methods, including both architectural and optimization based strategies.

show abstract

Section: Othermentioning

confidence: 99%

Multi-Task Learning for Dense Prediction Tasks: A Survey

Vandenhende¹,

Georgoulis²,

Gansbeke³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

428

222

View full text Add to dashboard Cite

show abstract

“…Diba et al [ 97 ] introduced a new spatio-temporal, deep neural network architecture named “Holistic Presence and Temporal Network” (HATNet) that builds on the convergence of 2D and 3D architectures into one by integrating intermediate representations of presence and temporal signals.…”

Section: Computer Vision Applicationsmentioning

confidence: 99%

Horizontal Review on Video Surveillance for Smart Cities: Edge Devices, Applications, Datasets, and Future Trends

Ezzat

Ghany

Almotairi

et al. 2021

Sensors

View full text Add to dashboard Cite

The automation strategy of today’s smart cities relies on large IoT (internet of Things) systems that collect big data analytics to gain insights. Although there have been recent reviews in this field, there is a remarkable gap that addresses four sides of the problem. Namely, the application of video surveillance in smart cities, algorithms, datasets, and embedded systems. In this paper, we discuss the latest datasets used, the algorithms used, and the recent advances in embedded systems to form edge vision computing are introduced. Moreover, future trends and challenges are addressed.

show abstract

“…For example, the user might want to search the specified moment of eating a hamburger, a piece of cake, or pizza. Diba et al [7] construct a dataset for video recognition which contains the labels of scene, object, action, event, attribute, and concepts for each short video snippet. However, the information about the subject and the object of the action is not provided.…”

Section: Related Workmentioning

confidence: 99%

Personal Knowledge Base Construction from Multimodal Data

Yen

Chang

Huang

et al. 2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

With the passage of time, people often have misty memories of their past experiences. Information recall support for people by collecting personal lifelogs is emerging. Recently, people tend to record their daily life via filming Video Weblog (VLog), which contains visual and audio data. These large scale multimodal data can be used to support information recall service that enables users to query their past experiences. The challenging issue is the semantic gap between the visual concept and the textual query. In this paper, we aim to extract personal life events from vlogs shared on YouTube and construct a personal knowledge base (PKB) for individuals. A multitask learning model is proposed to extract the components of personal life events, such as subjects, predicates and objects. The evaluation is performed on a video collection from three YouTubers who are English native speakers. Experimental results show our model achieves promising performance.

show abstract

Large Scale Holistic Video Understanding

Cited by 86 publications

References 52 publications

Multi-Task Learning for Dense Prediction Tasks: A Survey

Multi-Task Learning for Dense Prediction Tasks: A Survey

Horizontal Review on Video Surveillance for Smart Cities: Edge Devices, Applications, Datasets, and Future Trends

Personal Knowledge Base Construction from Multimodal Data

Contact Info

Product

Resources

About