Learned Initializations for Optimizing Coordinate-Based Neural Representations

Abstract

Coordinate-based neural representations have shown significant promise as an alternative to discrete, arraybased representations for complex low dimensional signals. However, optimizing a coordinate-based network from randomly initialized weights for each new signal is inefficient. We propose applying standard meta-learning algorithms to learn the initial weight parameters for these fully-connected networks based on the underlying class of signals being represented (e.g., images of faces or 3D models of chairs). Despite requiring only a minor change in implementation, using these learned initial weights enables faster convergence during optimization and can serve as a strong prior over the signal class being modeled, resulting in better generalization when only partial observations of a given signal are available. We explore these benefits across a variety of tasks, including representing 2D images, reconstructing CT scans, and recovering 3D shapes and scenes from 2D image observations.

COIN++: Neural Compression Across Modalities

“…Recently, Sitzmann et al (2020a); Tancik et al (2020a) have shown that applying MAML (Finn et al, 2017) to INRs can reduce fitting at test time to just a few gradient steps. Instead of minimizing L(θ, d) directly via gradient descent from a random initialization, we can meta-learn an initialization θ * such that minimizing L(θ, d) can be done in a few gradient steps.…”

COIN: COmpression with Implicit Neural representations

“…However, paying a significant computational cost upfront to compress content for delivery to many receivers is a standard practice in the setting of one-to-many media distribution, e.g., at Netflix [1]. Nevertheless, this limitation could likely be sidestepped with meta-learning [29,33] or amortized inference approaches. Further, at decoding time, we are required to evaluate the network at every pixel location to decode the full image.…”

“…Recent work in generative modeling of implicit representations [10] suggests that learning a distribution over the function weights could translate to significant compression gains for our approach. In addition, exploring meta-learning or other amortization approaches for faster encoding could be an important direction for future work [29,33]. Refining the architectures of the functions representing the images (through neural architecture search or pruning for example) is another promising avenue.…”

Neural radiosity

“…network to train on a novel domain, in this case a different scene. Other methods of handling dynamic scenes include meta-learning, in which we would optimize a model initialization that can quickly adapt to any general scene; similar meta-learning formulations have already been proposed [Tancik et al 2020a]. Other directions include few-shot learning, which would entail fine-tuning to a novel scene configuration given a very limited sample budget; in the context of our problem, we may design specialized sampling methods targeted at regions with changed geometry.…”