Jet-Based Local Image Descriptors

Larsen, Allan; Darkner, Sune; Dahl, Anders Bjorholm; Pedersen, Kim Steenstrup

doi:10.1007/978-3-642-33712-3_46

Cited by 21 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the spatial Gaussian scale-space concept [24,34,44,46,47,59,60,67,70,106,111,120,123], a general methodology for spatial scale selection has been developed based on local extrema over spatial scales of scale-normalized differential entities [62,64,65,72,73]. This general method- 2 The spatial Laplacian applied to the first-and second-order temporal derivatives ∇ 2 (x,y) L t and ∇ 2 (x,y) L tt as well as the spatio-temporal Laplacian ∇ 2 (x,y,t) L computed from a video sequence in the UCF-101 dataset (Kayaking_g01_c01.avi) at 3 × 3 combinations of the spatial scales (bottom row) σ s,1 = 2 pixels, (middle row) σ s,2 = 4.6 pixels and (top row) σ s,3 = 10.6 pixels and the temporal scales (left column) σ τ,1 = 40 ms, (middle column) σ τ,2 = 160 ms and (right column) σ τ,3 = 640 ms with the spatial and temporal scale parameters in units of σ s = √ s and σ τ = √ τ and using a time-causal spatio-temporal scale-space representation with a logarithmic distribution of the temporal scale levels for c = 2 (image size: 320 × 172 pixels of original 320 × 240 pixels; frame 90 of 226 frames at 25 framesframes/s) ology has in turn been successfully applied to develop robust methods for image-based matching and recognition [5,41,52,68,74,84,86,87,89,90,[112][113][114] that are able to handle large variations of the size of the objects in the image domain and with numerous applications regarding object recognition, object categorization, multi-view geometry, construction of 3-D models from visual input,…”

Section: Figmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

View full text Add to dashboard Cite

This work presents a theory and methodology for simultaneous detection of local spatial and temporal scales in video data. The underlying idea is that if we process video data by spatio-temporal receptive fields at multiple spatial and temporal scales, we would like to generate hypotheses about the spatial extent and the temporal duration of the underlying spatio-temporal image structures that gave rise to the feature responses. For two types of spatio-temporal scale-space representations, (i) a non-causal Gaussian spatio-temporal scale space for offline analysis of pre-recorded video sequences and (ii) a time-causal and timerecursive spatio-temporal scale space for online analysis of real-time video streams, we express sufficient conditions for spatio-temporal feature detectors in terms of spatio-temporal receptive fields to deliver scale-covariant and scale-invariant feature responses. We present an in-depth theoretical analysis of the scale selection properties of eight types of spatio-temporal interest point detectors in terms of either: (i)-(ii) the spatial Laplacian applied to the first-and secondorder temporal derivatives, (iii)-(iv) the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives, (v) the determinant of the spatio-temporal Hessian matrix, (vi) the spatio-temporal Laplacian and (vii)-(viii) the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix. It is shown that seven of these spatio-temporal feature detectors allow for provable scale covariance and scale invariance. Then, we describe a time-causal and time-recursive algorithm for detecting sparse spatio-temporal interest points from video streams and show that it leads to intuitively reasonable results. An experimental quantification of the accuracy of the spatio-temporal scale estimates and the amount of temporal delay obtained from these spatio-temporal interest point detectors is given, showing that: (i) the spatial and temporal scale selection properties predicted by the continuous theory are well preserved in the discrete implementation and (ii) the spatial Laplacian or the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives leads to much shorter temporal delays in a timecausal implementation compared to the determinant of the spatio-temporal Hessian or the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix.

show abstract

Section: Figmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

View full text Add to dashboard Cite

show abstract

“…Dense local approaches have been investigated by Jurie and Triggs [62], Lazebnik et al [84], Bosch et al [18], Agarwal and Triggs [2] and Tola et al [150]. More recently, Larsen et al [81] made use of multi-local N-jet descriptors that do not rely on a spatial statistics of receptive field responses as used in the SIFT and SURF descriptors or their analogues. A notable observation from experimental results is that very good performance can be obtained with coarsely quantized even binary image descriptors (Pietikäinen et al [131], Linde and Lindeberg [88], Calonder et al [26]).…”

Section: Related Workmentioning

confidence: 99%

“…[7]) have been demonstrated to be highly useful for this purpose with many successful applications, including multi-view image matching, object recognition, 3-D object and scene modelling, video tracking, gesture recognition, panorama stitching as well as robot localization and mapping. Different generalizations of the SIFT operator in terms of the image descriptor have been presented by Ke and Sukthankar [66], Mikolajczyk and Schmid [125], Burghouts and Geusebroek [24], Toews and Wells [149], van de Sande et al [138], Tola et al [150] and Larsen et al [81].…”

mentioning

confidence: 99%

Image Matching Using Generalized Scale-Space Interest Points

Lindeberg

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian ∇ 2 norm L, the determinant of the Hessian det H norm L and four new unsigned or signed Hessian feature strength measures D 1,norm L,D 1,norm L, D 2,norm L and D 2,norm L, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian det H norm L is a truly affine covariant differential entity and the Hessian feature strength measures D 1,norm L andD 1,norm L have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differen- It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures D 1,norm L,D 1,norm L and the determinant of the Hessian det H norm L being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris-Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.

show abstract

“…Unfortunately, this approach can only handle moderate changes in illumination, and failed on our test sequences. While they have never been used for direct image alignment -to the best of our knowledge-it seems interesting to use "local jets" for the d function [4,22,10,11]. Local jets are vectors often used as local descriptors and efficiently computed by convolving an image with a series of filters:…”

Section: Descriptor Fieldsmentioning

confidence: 99%

Robust 3D Tracking with Descriptor Fields

Crivellaro

Lepetit

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

We introduce a method that can register challenging images from specular and poorly textured 3D environments, on which previous approaches fail. We assume that a small set of reference images of the environment and a partial 3D model are available. Like previous approaches, we register the input images by aligning them with one of the reference images using the 3D information. However, these approaches typically rely on the pixel intensities for the alignment, which is prone to fail in presence of specularities or in absence of texture. Our main contribution is an efficient novel local descriptor that we use to describe each image location. We show that we can rely on this descriptor in place of the intensities to significantly improve the alignment robustness at a minor increase of the computational cost, and we analyze the reasons behind the success of our descriptor.

show abstract

Jet-Based Local Image Descriptors

Cited by 21 publications

References 20 publications

Spatio-Temporal Scale Selection in Video Data

Spatio-Temporal Scale Selection in Video Data

Image Matching Using Generalized Scale-Space Interest Points

Robust 3D Tracking with Descriptor Fields

Contact Info

Product

Resources

About