Untrimmed Action Anticipation

Rodin, Ivan; Furnari, Antonino; Mavroeidis, Dimitrios; Farinella, Giovanni Maria

doi:10.1007/978-3-031-06433-3_29

Cited by 9 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most approaches assume that a "trimmed" video is sampled at a fixed time before the beginning of the action and fed to the model, which constitutes an unrealistic scenario, given that the occurrence of future actions is unknown at test time. Despite some recent work towards an untrimmed anticipation scenario (Rodin et al, 2022), the trimmed setting remains the most common one.…”

Section: State-of-the-art Papersmentioning

confidence: 99%

An Outlook into the Future of Egocentric Vision

Plizzari,

Goletto,

Furnari

et al. 2024

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

What will the future be? We wonder! In this survey, we explore the gap between current research in egocentric vision and the ever-anticipated future, where wearable computing, with outward facing cameras and digital overlays, is expected to be integrated in our every day lives. To understand this gap, the article starts by envisaging the future through character-based stories, showcasing through examples the limitations of current technology. We then provide a mapping between this future and previously defined research tasks. For each task, we survey its seminal works, current state-of-the-art methodologies and available datasets, then reflect on shortcomings that limit its applicability to future research. Note that this survey focuses on software models for egocentric vision, independent of any specific hardware. The paper concludes with recommendations for areas of immediate explorations so as to unlock our path to the future always-on, personalised and life-enhancing egocentric vision.

show abstract

Section: State-of-the-art Papersmentioning

confidence: 99%

An Outlook into the Future of Egocentric Vision

Plizzari,

Goletto,

Furnari

et al. 2024

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, (Rodin et al 2022) tried to fine-tune trimmed methods for untrimmed videos for action prediction, but got poor results. We argue they ignored the long-tail distribution what is common in untrimmed videos.…”

Section: Intention Understanding Based On Untrimmed Videosmentioning

confidence: 99%

“…This is due to the fact that videos are generally untrimmed. Recently, (Rodin et al 2022) attempted to fine-tune trimmed methods to untrimmed videos and concluded that "perform-ing action prediction tasks on untrimmed videos is challenging". We believe that one reason for the unsatisfactory results is that it ignores the long-tail distribution.…”

Section: Introductionmentioning

confidence: 99%

Intentional Evolutionary Learning for Untrimmed Videos with Long Tail Distribution

Zhou,

Wang,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

Human intention understanding in untrimmed videos aims to watch a natural video and predict what the person’s intention is. Currently, exploration of predicting human intentions in untrimmed videos is far from enough. On the one hand, untrimmed videos with mixed actions and backgrounds have a significant long-tail distribution with concept drift characteristics. On the other hand, most methods can only perceive instantaneous intentions, but cannot determine the evolution of intentions. To solve the above challenges, we propose a loss based on Instance Confidence and Class Accuracy (ICCA), which aims to alleviate the prediction bias caused by the long-tail distribution with concept drift characteristics in video streams. In addition, we propose an intention-oriented evolutionary learning method to determine the intention evolution pattern (from what action to what action) and the time of evolution (when the action evolves). We conducted extensive experiments on two untrimmed video datasets (THUMOS14 and ActivityNET v1.3), and our method has achieved excellent results compared to SOTA methods. The code and supplementary materials are available at https://github.com/Jennifer123www/UntrimmedVideo.

show abstract

“…We opted for a trimmed scenario to deviate as little as possible from the standard definition of short-term egocentric action anticipation [8], which allows us to better investigate the impact of assuming a streaming scenario. The reader is referred to [58] for a thorough investigation of untrimmed action anticipation. (b) Illustration of the quantization formula reported in Eq.…”

Section: Proposed Streaming Evaluation Schemementioning

confidence: 99%

“…In particular DIST-R(2+1)D-S obtains an offline action accuracy of 7.77 (+2. 58 1, methods with a smaller runtime experience a smaller performance gap when passing from an offline to a streaming evaluation scenario. For instance, LSTM passes from an Action Mean Top-5 Recall of 12.49 in offline settings to 12.38 in the streaming scenario (only −0.11).…”

Section: Epic-kitchens-100mentioning

confidence: 99%