Spatio-temporal Scale-Spaces

Abstract

Abstract. A family of spatio-temporal scale-spaces suitable for a moving observer is developed. The scale-spaces are required to be time causal for being usable for real time measurements, and to be "velocity adapted", i.e. to have Galilean covariance to avoid favoring any particular motion. Furthermore standard scale-space axioms: linearity, positivity, continuity, translation invariance, scaling covariance in space and time, rotational invariance in space and recursivity are used. An infinitesimal criterion for scale-spaces is developed, which simplifies calculations and makes it possible to define scale spaces on bounded regions. We show that there are no temporally causal Galilean scale-spaces that are semigroups acting on space and time, but that there are such scale-spaces that are semigroups acting on space and memory (where the memory is the scale-space). The temporally causal scale-space is a time-recursive process using current input and the scale-space as state, i.e. there is no need for storing earlier input. The diffusion equation acting on the memory with the input signal as boundary condition, is a member of this family of scale spaces and is special in the sense that its generator is local.

Generalized Gaussian Scale-Space Axiomatics Comprising Linear Scale-Space, Affine Scale-Space and Spatio-Temporal Scale-Space

“…This form of time-causal spatio-temporal scale-space has also been derived by Fagerström [2007] in the special case when Σ = I, however, starting from different arguments of scale invariance. The additional degree of freedom in the spatial covariance matrix Σ obtained 7 If the full group of spatial covariance matrices Σ and velocity vectors v is considered as well, the dimensionality of the affine-and velocity-adapted scale-space will be dim(x) + dim(t) + dim(Σ) + dim(v) + dim(τ ) = N + 1 + N (N + 1)/2 + N + 1 = (N 2 +5N +4)/2.…”

“…Early work on non-separable spatio-temporal scale-spaces with velocity adaptation was presented in Lindeberg [1997bLindeberg [ , 2002 with applications to Galilean invariant image descriptors and recognition of activities in [Laptev and Lindeberg, 2004, Lindeberg et al, 2004b, Laptev et al, 2007. More recently, Fagerström [2005Fagerström [ , 2007 has studied scale-invariant continuous scale-space models that allows for the construction of continuous semi-groups over the internal memory representation and in a special case lead to a diffusion formulation.…”

“…Furthermore, following Fagerström [2007] we are seeking a scale-space representation that at any time moment to be computed from a continuous semi-group structure over time. Thus, given any spatio-temporal signal f (x, t) over space-time (x, t), we embed this signal into a scale-space representation L using a complementary spatial scale parameter s and a complementary temporal scale parameter τ that it is to be updated according to the following time-recursive formulation…”

Improved Spatio-temporal Salient Feature Detection for Action Recognition

“…According to scale-space requirements for real-time multi-resolution video representation [6,7,8,15,18,31], the diffusion in the time direction should be limited to just the past frames to respect the time causality observed in the biological vision. For a discrete signal, Lindeberg et al [18] obtained a Poisson kernel as a canonical time-causal scale-space kernel by cascading a set of truncated exponential kernels.…”

SpaceTime Environmental Image Information for Scene Understanding

“…[39][40][41] Nevertheless, camera motion may introduce some unmanageable artifacts with some of these gradient-based optical flow approaches if they are not augmented by more sophisticated spatiotemporal analyses. [42][43][44] In this report, we propose that it is important to incorporate space-and time-varying environmental image information from the very beginning of the data collection process so that the recorded images can be more effectively indexed and retrieved for operational use and analysis. This top-down approach not only provides a systematic characterization of the measured data for better scene description, but can help the end user (Soldier) develop improved course of action strategies based on scene understanding (algorithms and analysis) incorporating battlefield environments changing in space and time.…”