Efficient Graph Construction For Image Representation

Shekkizhar, Sarath; Ortega, Antonio

doi:10.1109/icip40778.2020.9191232

Cited by 9 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, NNK neighborhoods are stable for large enough K, and their size is indicative of the ID of the manifold the data belongs to [15]. This method has been shown to provide advantages for semi-supervised learning, image representation [16], and generalization estimation [10]. We first proposed a channel-wise approach for NNK graphs in [17], where information from multiple channels was used to estimate generalization and perform early stopping without requiring a validation set.…”

Section: Introductionmentioning

confidence: 99%

Channel Redundancy and Overlap in Convolutional Neural Networks with Channel-Wise NNK Graphs

Bonet

Ortega

Ruiz-Hidalgo

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Feature spaces in the deep layers of convolutional neural networks (CNNs) are often very high-dimensional and difficult to interpret. However, convolutional layers consist of multiple channels that are activated by different types of inputs, which suggests that more insights may be gained by studying the channels and how they relate to each other. In this paper, we first analyze theoretically channelwise non-negative kernel (CW-NNK) regression graphs, which allow us to quantify the overlap between channels and, indirectly, the intrinsic dimension of the data representation manifold. We find that redundancy between channels is significant and varies with the layer depth and the level of regularization during training. Additionally, we observe that there is a correlation between channel overlap in the last convolutional layer and generalization performance. Our experimental results demonstrate that these techniques can lead to a better understanding of deep representations.

show abstract

Section: Introductionmentioning

confidence: 99%

Channel Redundancy and Overlap in Convolutional Neural Networks with Channel-Wise NNK Graphs

Bonet

Ortega

Ruiz-Hidalgo

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…NNK has been shown to perform well in several machine learning tasks [15], image representation [16], and generalization estimation in neural networks [17]. Furthermore, NNK has also been used to understand convolutional neural networks (CNN) channel redundancy [18] and to propose an early stopping criterion for them [19].…”

Section: Non-negative Kernel (Nnk) Regression Graphsmentioning

confidence: 99%

Study of Manifold Geometry Using Multiscale Non-Negative Kernel Graphs

Hurtado

Shekkizhar

Ruiz-Hidalgo

et al. 2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Modern machine learning systems are increasingly trained on large amounts of data embedded in high-dimensional spaces. Often this is done without analyzing the structure of the dataset. In this work, we propose a framework to study the geometric structure of the data. We make use of our recently introduced non-negative kernel (NNK) regression graphs to estimate the point density, intrinsic dimension, and linearity of the data manifold (curvature). We further generalize the graph construction and geometric estimation to multiple scales by iteratively merging neighborhoods in the input data. Our experiments demonstrate the effectiveness of our proposed approach over other baselines in estimating the local geometry of the data manifolds on synthetic and real datasets.

show abstract

“…NNK has been shown to deliver good results in several machine learning tasks [15], image representation [16], and generalization estimation in neural networks [17]. Furthermore, NNK has also been used to understand convolutional neural networks (CNN) channel redundancy [18] and to propose an early stopping criterion for them [19].…”

Section: Non-negative Kernel (Nnk) Regression Graphsmentioning

confidence: 99%

Study of Manifold Geometry using Multiscale Non-Negative Kernel Graphs

Hurtado¹,

Shekkizhar²,

Ruiz-Hidalgo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Modern machine learning systems are increasingly trained on large amounts of data embedded in high-dimensional spaces. Often this is done without analyzing the structure of the dataset. In this work, we propose a framework to study the geometric structure of the data. We make use of our recently introduced non-negative kernel (NNK) regression graphs to estimate the point density, intrinsic dimension, and the linearity of the data manifold (curvature). We further generalize the graph construction and geometric estimation to multiple scale by iteratively merging neighborhoods in the input data. Our experiments demonstrate the effectiveness of our proposed approach over other baselines in estimating the local geometry of the data manifolds on synthetic and real datasets.

show abstract

Efficient Graph Construction For Image Representation

Cited by 9 publications

References 19 publications

Channel Redundancy and Overlap in Convolutional Neural Networks with Channel-Wise NNK Graphs

Channel Redundancy and Overlap in Convolutional Neural Networks with Channel-Wise NNK Graphs

Study of Manifold Geometry Using Multiscale Non-Negative Kernel Graphs

Study of Manifold Geometry using Multiscale Non-Negative Kernel Graphs

Contact Info

Product

Resources

About