A Structured Dictionary Perspective on Implicit Neural Representations

Yuce, Gizem; Ortiz-Jiménez, Guillermo; Besbinar, Beril; Frossard, Pascal

doi:10.1109/cvpr52688.2022.01863

Cited by 42 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fathony et al propose MFN [7], which employs a Hadamard product between linear layers and nonlinear activation functions. In [62], Yüce et al prove that FFN [47], SIREN [42] and MFN [7] have the same expressive power as a structured signal dictionary. 3D Representations for View Synthesis.…”

Section: Related Workmentioning

confidence: 99%

“…Radiance mapping, also known as radiance field, is a type of Implicit Neural Representation (INR). There have been some studies [2,7,32,42,46,47,62] on the expressive power and inductive bias of INRs. The standard Multi-layer Perceptrons (MLPs) with ReLU activation function are well known for the strong spectral bias towards reconstructing low frequency signals [32].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Frequency-Modulated Point Cloud Rendering with Easy Editing

Zhang¹,

Huang²,

Ni³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Frequency-Modulated Point Cloud Rendering with Easy Editing

Zhang¹,

Huang²,

Ni³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…As stated, we seek an INR that fits visual signals well but fits noise poorly in comparison. Inspired by [53], who proposed to compare eigenfunctions of the empirical neural tangent kernel (NTK) [22] of INRs to understand their approximation properties, we compare the fitting of noisy natural images using NTK gradient flow. The NTK gradient flow of INRs accurately captures the behavior of early training of neural networks, and so in tasks such as denoising where we regularize via early stopping, the early training behavior determines the implicit bias.…”

Section: Implicit Bias Of Wirementioning

confidence: 99%

“…To achieve this, we take inspiration from harmonic analysis and reconsider the nonlinear activation function used in the MLP. Recent work has shown that an INR can be interpreted as a structured signal representation dictionary [53], where the activation nonlinearity dictates the atoms of the dictionary. For example, the sine activation creates a pseudo-Fourier transform representation of the signal that is maximally concentrated in the frequency domain [53].…”

Section: Introductionmentioning

confidence: 99%

WIRE: Wavelet Implicit Neural Representations

Saragadam¹,

LeJeune²,

Tan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Implicit neural representations (INRs) have recently advanced numerous vision-related areas. INR performance depends strongly on the choice of the nonlinear activation function employed in its multilayer perceptron (MLP) network. A wide range of nonlinearities have been explored, but, unfortunately, current INRs designed to have high accuracy also suffer from poor robustness (to signal noise, parameter variation, etc.). Inspired by harmonic analysis, we develop a new, highly accurate and robust INR that does not exhibit this tradeoff. Wavelet Implicit neural REpresentation (WIRE) uses a continuous complex Gabor wavelet activation function that is well-known to be optimally concentrated in space-frequency and to have excellent biases for representing images. A wide range of experiments (image denoising, image inpainting, super-resolution, computed tomography reconstruction, image overfitting, and novel view synthesis with neural radiance fields) demonstrate that WIRE defines the new state of the art in INR accuracy, training time, and robustness.

show abstract

“…SIREN depends on periodic activation functions, i.e., sinusoidal activations, to continuously represent the signals with fine details. Both FFN and SIREN are efficient, and some work [19,20] have proven that they are equivalent to each other. INRs have been adopted for many computer vision tasks, including neural radiance fields for novel view synthesizing [21], image generation [22], unconditional video generation [23] and video interpolation [24].…”

Section: Introductionmentioning

confidence: 99%

Continuous conditional video synthesis by neural processes

Yang¹,

Bilodeau²

2022

Preprint

View full text Add to dashboard Cite

We propose a unified model for multiple conditional video synthesis tasks, including video prediction and video frame interpolation. We show that conditional video synthesis can be formulated as a neural process, which maps input spatio-temporal coordinates to target pixel values given context spatio-temporal coordinates and pixels values. Specifically, we feed an implicit neural representations of coordinates into a Transformer-based non-autoregressive conditional video synthesis model. Our task-specific models outperform previous work for video interpolation on multiple datasets and reach a competitive performance with the state-of-the-art models for video prediction. Importantly, the model is able to interpolate or predict with an arbitrary high frame rate, i.e., continuous synthesis. Our source code is available at https://github.com/NPVS/NPVS.

show abstract

A Structured Dictionary Perspective on Implicit Neural Representations

Cited by 42 publications

References 27 publications

Frequency-Modulated Point Cloud Rendering with Easy Editing

Frequency-Modulated Point Cloud Rendering with Easy Editing

WIRE: Wavelet Implicit Neural Representations

Continuous conditional video synthesis by neural processes

Contact Info

Product

Resources

About