Image Morphing in Deep Feature Spaces: Theory and Applications

Effland, Alexander; Kobler, Erich; Pock, Thomas; Rajković, Marko; Rumpf, Martin

doi:10.1007/s10851-020-00974-5

“…Note that a morphing process happens between our two images as depicted in Figure 8 which is different than image interpolation [Lakshman et al, 2015]. Hence, our formulations can be used for image morphing which has its own literature in image processing and deep learning [Effland et al, 2021]. Moreover, this transformation is not linear, i.e., change does not occur at a linear rate along the path between the two images.…”

Section: Support In the Training Setmentioning

confidence: 99%

Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

Yousefzadeh¹

2022

Preprint

View full text Add to dashboard Cite

The success of deep neural networks in image classification and learning can be partly attributed to the features they extract from images. It is often speculated about the properties of a low-dimensional manifold that models extract and learn from images. However, there is not sufficient understanding about this low-dimensional space based on theory or empirical evidence. For image classification models, their last hidden layer is the one where images of each class is separated from other classes and it also has the least number of features. Here, we develop methods and formulations to study that feature space for any model. We study the partitioning of the domain in feature space, identify regions guaranteed to have certain classifications, and investigate its implications for the pixel space. We observe that geometric arrangements of decision boundaries in feature space is significantly different compared to pixel space, providing insights about adversarial vulnerabilities, image morphing, extrapolation, ambiguity in classification, and the mathematical understanding of image classification models.

show abstract

“…Note that a morphing process happens between our two images as depicted in Figure 8 which is different than image interpolation (Lakshman et al, 2015). Hence, our formulations can be used for image morphing which has its own literature in image processing and deep learning (Effland et al, 2021). Moreover, this transformation is not linear, i.e., change does not occur at a linear rate along the path between the two images.…”

Section: Support In the Training Setmentioning

confidence: 99%

Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

Yousefzadeh

¹

2022

Preprint

1

0

View full text Add to dashboard Cite

The success of deep neural networks in image classification and learning can be partly attributed to the features they extract from images. It is often speculated about the properties of a low-dimensional manifold that models extract and learn from images. However, there is not sufficient understanding about this low-dimensional space based on theory or empirical evidence. For image classification models, their last hidden layer is the one where images of each class is separated from other classes and it also has the least number of features. Here, we develop methods and formulations to study that feature space for any model. We study the partitioning of the domain in feature space, identify regions guaranteed to have certain classifications, and investigate its implications for the pixel space. We observe that geometric arrangements of decision boundaries in feature space is significantly different compared to pixel space, providing insights about adversarial vulnerabilities, image morphing, extrapolation, ambiguity in classification, and the mathematical understanding of image classification models.

show abstract

“…Neural Networks (NN) have been acknowledged to be a very powerful tool in machine learning tasks: A not fully comprehensive list of such tasks includes as speech-to-text transcription [1,2], image segmentation [3,4], image classification [5], match new items and/or products with user's interests [6], image morphing [7], imitation learning [8], solution to nonlinear PDEs [9,10], image generation [11,12]. The beginning of the new millennium has seen a growing interest in this automatic learning approach, due to the rising computational power (more performant GPUs) and the huge amount of data (the so-called Big Data Revolution).…”

Section: Introductionmentioning

confidence: 99%

A singular Riemannian geometry approach to Deep Neural Networks II. Reconstruction of 1-D equivalence classes

Benfenati¹,

Marta²

2021

Preprint

0

View full text Add to dashboard Cite

show abstract

Image Morphing in Deep Feature Spaces: Theory and Applications

Abstract: This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a highdimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller and Younes (

Cited by 8 publications

References 29 publications

Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

Decision boundaries and convex hulls in the feature space that deep learning functions learn from images

A singular Riemannian geometry approach to Deep Neural Networks II. Reconstruction of 1-D equivalence classes

Contact Info

Product

Resources

About