Local velocity-adapted motion events for spatio-temporal recognition

Laptev, Ivan; Caputo, Barbara; Schüldt, Christian; Lindeberg, Tony

doi:10.1016/j.cviu.2006.11.023

Cited by 129 publications

(114 citation statements)

References 56 publications

Supporting

Mentioning

110

Contrasting

Unclassified

Order By: Relevance

“…An alternative way of handling spatio-temporal scenes with dominant relative motions between the camera and the environment, in contrast to this use of space-time separable receptive fields for only image velocity v = 0, is by exploiting the full structure of the spatio-temporal receptive field model (1), by considering spatio-temporal receptive fields with nonzero image velocities v = 0, which can be locally adapted to the local motion direction corresponding to velocity adaptation [50,51,61] or alternatively performing local, regional or global image stabilization. Then, the image operations can be made truly covariant under local, regional or global Galilean image transformations [67,71] and allow for a more explicit separation of spatio-temporal receptive field responses that correspond to more complex spatio-temporal image structures than local Galilean motions.…”

Section: Resultsmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

Self Cite

View full text Add to dashboard Cite

This work presents a theory and methodology for simultaneous detection of local spatial and temporal scales in video data. The underlying idea is that if we process video data by spatio-temporal receptive fields at multiple spatial and temporal scales, we would like to generate hypotheses about the spatial extent and the temporal duration of the underlying spatio-temporal image structures that gave rise to the feature responses. For two types of spatio-temporal scale-space representations, (i) a non-causal Gaussian spatio-temporal scale space for offline analysis of pre-recorded video sequences and (ii) a time-causal and timerecursive spatio-temporal scale space for online analysis of real-time video streams, we express sufficient conditions for spatio-temporal feature detectors in terms of spatio-temporal receptive fields to deliver scale-covariant and scale-invariant feature responses. We present an in-depth theoretical analysis of the scale selection properties of eight types of spatio-temporal interest point detectors in terms of either: (i)-(ii) the spatial Laplacian applied to the first-and secondorder temporal derivatives, (iii)-(iv) the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives, (v) the determinant of the spatio-temporal Hessian matrix, (vi) the spatio-temporal Laplacian and (vii)-(viii) the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix. It is shown that seven of these spatio-temporal feature detectors allow for provable scale covariance and scale invariance. Then, we describe a time-causal and time-recursive algorithm for detecting sparse spatio-temporal interest points from video streams and show that it leads to intuitively reasonable results. An experimental quantification of the accuracy of the spatio-temporal scale estimates and the amount of temporal delay obtained from these spatio-temporal interest point detectors is given, showing that: (i) the spatial and temporal scale selection properties predicted by the continuous theory are well preserved in the discrete implementation and (ii) the spatial Laplacian or the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives leads to much shorter temporal delays in a timecausal implementation compared to the determinant of the spatio-temporal Hessian or the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix.

show abstract

Section: Resultsmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

Self Cite

View full text Add to dashboard Cite

show abstract

“…This match kernel has been used in object recognition [27] and action classification [64]. Lyu et al [26] has proven it to be a non-mercer kernel, and proposed a normalized sum-match kernel which satisfies the mercer condition and is defined as follows:…”

Section: Match Kernelsmentioning

confidence: 99%

Action recognition via spatio-temporal local features: A comprehensive study

Zhen

Shao

2016

Image and Vision Computing

View full text Add to dashboard Cite

Local methods based on spatio-temporal interest points (STIPs) have shown their effectiveness for human action recognition. The bag-of-words (BoW) model has been widely used and dominated in this field. Recently, a large number of techniques based on local features including improved variants of the BoW model, sparse coding (SC), Fisher kernels (FK), vector of locally aggregated descriptors (VLAD) as well as the naive Bayes nearest neighbor (NBNN) classifier have been proposed and developed for visual recognition. However, some of them are proposed in the image domain and have not yet been applied to the video domain and it is still unclear how effectively these techniques would perform on action recognition. In this paper, we provide a comprehensive study on these local methods for human action recognition. We implement these techniques and conduct comparison under unified experimental settings on three widely used benchmarks, i.e., the KTH, UCF-YouTube and HMDB51 datasets. We discuss insightfully the findings from the experimental results and draw useful conclusions, which are expected to guide practical applications and future work for the action recognition community.

show abstract

“…One of the few interest point detectors of a spatiotemporal nature is an extension of the Harris corner detection to 3D, which has been studied quite extensively by Laptev and Lindeberg (2003) and further developed and used in Laptev et al (2007). A spatio-temporal corner is defined as an image region containing a spatial corner whose velocity vector is changing direction.…”

Section: Action Recognitionmentioning

confidence: 99%

Vision, Attention Control, and Goals Creation System

Rapantzikos

Avrithis²,

Kolias³

2010

Perception-Action Cycle

View full text Add to dashboard Cite

Biological visual attention has been long studied by experts in the field of cognitive psychology. The Holy Grail of this study is the exact modeling of the interaction between the visual sensory and the process of perception. It seems that there is an informal agreement on the four important functions of the attention process: (a) the bottom-up process, which is responsible for the saliency of the input stimuli; (b) the top-down process that bias attention toward known areas or regions of predefined characteristics; (c) the attentional selection that fuses information derived from the two previous processes and enables focus; and (d) the dynamic evolution of the attentional selection process. In the following, we will outline established computational solutions for each of the four functions. OverviewMost of our impressions and memories are based on vision. Nevertheless vision mechanisms and functionalities are still not apparent. How do we perceive shape, color, or motion and how do we automatically focus on the most informative parts of the visual input? It has been long established that primates, including human, use focused attention and fast saccades to analyze visual stimuli based on the current situation or the desired goal. Neuroscientists have proven that neural information related to shape, motion, and color is transmitted through, at least, three parallel and interconnected channels to the brain rather than a single one. Hence a second question arises related to how these channels are "linked" in order to provide useful information to the brain.The Human Visual System (HVS) creates a perceptual representation of the world that is quite different than the two dimensional depiction of the retina.K. Rapantzikos ( ) Image, Video and

show abstract

Local velocity-adapted motion events for spatio-temporal recognition

Cited by 129 publications

References 56 publications

Spatio-Temporal Scale Selection in Video Data

Spatio-Temporal Scale Selection in Video Data

Action recognition via spatio-temporal local features: A comprehensive study

Vision, Attention Control, and Goals Creation System

Contact Info

Product

Resources

About