A computational theory of visual receptive fields

Lindeberg, Tony

doi:10.1007/s00422-013-0569-z

Cited by 114 publications

(163 citation statements)

References 177 publications

(373 reference statements)

Supporting

Mentioning

155

Contrasting

Unclassified

Order By: Relevance

“…Let us assume a (i) perspective camera model extended with (ii) a thin circular lens for gathering incoming light from different directions and (iii) a Lambertian illumination model extended with (iv) a spatially varying albedo factor for modelling the light that is reflects from surface patterns in the world. Then, it can be shown (Lindeberg [106,Sect. 2.3]) that a spatial receptive field response…”

Section: Extension To Illumination Invariancementioning

confidence: 99%

“…A general theoretical framework for how local receptive field responses, as used in the SIFT and SURF descriptors and their extensions or analogues to colour images and spatiotemporal image data, can constitute the basis for computing inherent properties of objects to support invariant recognition under natural image transformations is presented in (Lindeberg [106,109]) including relations to receptive fields in biological vision.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Image Matching Using Generalized Scale-Space Interest Points

Lindeberg

2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian ∇ 2 norm L, the determinant of the Hessian det H norm L and four new unsigned or signed Hessian feature strength measures D 1,norm L,D 1,norm L, D 2,norm L and D 2,norm L, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian det H norm L is a truly affine covariant differential entity and the Hessian feature strength measures D 1,norm L andD 1,norm L have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differen- It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures D 1,norm L,D 1,norm L and the determinant of the Hessian det H norm L being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris-Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.

show abstract

Section: Extension To Illumination Invariancementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Image Matching Using Generalized Scale-Space Interest Points

Lindeberg

2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Based on this spatio-temporal receptive field model, we define a spatio-temporal scale-space representation of the form [67,69,75] …”

Section: Spatio-temporal Receptive Field Modelmentioning

confidence: 99%

“…Cell recordings from neurones in the primary visual cortex have shown that there are spatio-temporal receptive fields tuned to different sizes and orientations in the image domain, to different integration times over the temporal domain as well as to different image velocities in space-time [12,13,32,33]. Interestingly, the shapes of the spatio-temporal receptive field families that have been measured in biological vision can furthermore be explained by normative theories of visual receptive fields [69,71,75,78], whose axiomatic derivation is based on structural properties of the environment in combination with assumptions about the internal structure of an idealized vision system to ensure a consistent treatment of image representations over multiple spatio-temporal scales.…”

Section: Introductionmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

Self Cite

View full text Add to dashboard Cite

This work presents a theory and methodology for simultaneous detection of local spatial and temporal scales in video data. The underlying idea is that if we process video data by spatio-temporal receptive fields at multiple spatial and temporal scales, we would like to generate hypotheses about the spatial extent and the temporal duration of the underlying spatio-temporal image structures that gave rise to the feature responses. For two types of spatio-temporal scale-space representations, (i) a non-causal Gaussian spatio-temporal scale space for offline analysis of pre-recorded video sequences and (ii) a time-causal and timerecursive spatio-temporal scale space for online analysis of real-time video streams, we express sufficient conditions for spatio-temporal feature detectors in terms of spatio-temporal receptive fields to deliver scale-covariant and scale-invariant feature responses. We present an in-depth theoretical analysis of the scale selection properties of eight types of spatio-temporal interest point detectors in terms of either: (i)-(ii) the spatial Laplacian applied to the first-and secondorder temporal derivatives, (iii)-(iv) the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives, (v) the determinant of the spatio-temporal Hessian matrix, (vi) the spatio-temporal Laplacian and (vii)-(viii) the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix. It is shown that seven of these spatio-temporal feature detectors allow for provable scale covariance and scale invariance. Then, we describe a time-causal and time-recursive algorithm for detecting sparse spatio-temporal interest points from video streams and show that it leads to intuitively reasonable results. An experimental quantification of the accuracy of the spatio-temporal scale estimates and the amount of temporal delay obtained from these spatio-temporal interest point detectors is given, showing that: (i) the spatial and temporal scale selection properties predicted by the continuous theory are well preserved in the discrete implementation and (ii) the spatial Laplacian or the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives leads to much shorter temporal delays in a timecausal implementation compared to the determinant of the spatio-temporal Hessian or the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix.

show abstract

“…Such a framework then leads to Gaussian-related kernels characterizing any linear visual system including early biological visual systems when they behave linearly (see e.g. Weickert et al 1999;Lindeberg 2011;Lindeberg 2013;ter Haar Romeny et al 2001;ter Haar Romeny 2003;Koenderink 1988;Florack 1997). On the other hand, a model based on the anatomical and physiological properties of biological visual systems is proposed in (Mahmoodi 2015) to derive Gaussian-related kernels in spatial as well as spatio-temporal domains.…”

Section: Introductionmentioning

confidence: 99%

Nonlinear Model for Complex Neurons in Biological Visual Visions

Mahmoodi

Saba

2016

Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies

View full text Add to dashboard Cite

Abstract:Complex cells in biological visual vision are well known to be nonlinear. In this paper, it is demonstrated that these nonlinear complex cells can be modelled under some certain conditions by a biologically inspired model which is nonlinear in nature. Our model consists of cascaded neural layers accounting for anatomical evidence in biological early visual visions. In the model proposed in this paper, the axons associated with the complex cells are considered to operate nonlinearly. We also consider the second order interaction receptive maps as directional derivatives of the complex cell's kernel along the direction of orientation tuning. Our numerical results are similar to the biologically recorded data reported in the literature.

show abstract

A computational theory of visual receptive fields

Cited by 114 publications

References 177 publications

Image Matching Using Generalized Scale-Space Interest Points

Image Matching Using Generalized Scale-Space Interest Points

Spatio-Temporal Scale Selection in Video Data

Nonlinear Model for Complex Neurons in Biological Visual Visions

Contact Info

Product

Resources

About