COIN++: Neural Compression Across Modalities

Abstract

Neural compression algorithms are typically based on autoencoders that require specialized encoder and decoder architectures for different data modalities. In this paper, we propose COIN++, a neural compression framework that seamlessly handles a wide range of data modalities. Our approach is based on converting data to implicit neural representations, i.e. neural functions that map coordinates (such as pixel locations) to features (such as RGB values). Then, instead of storing the weights of the implicit neural representation directly, we store modulations applied to a meta-learned base network as a compressed code for the data. We further quantize and entropy code these modulations, leading to large compression gains while reducing encoding time by two orders of magnitude compared to baselines. We empirically demonstrate the effectiveness of our method by compressing various data modalities, from images to medical and climate data.

Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations

“…Further speedups for Neural Collages compressors could similarly be found via tracking during training. We note concurrent work on an improved version of COIN (Dupont et al, 2022) that is trained in parallel on patches, similarly to Neural Collage compressors.…”

Relationship of Neural Correlates of Gait Characteristics and Cognitive Dysfunction in Patients With Mild Cognitive Impairment

“…is attributed to its significant impact on climate prediction, mitigation, and adaptation. Employing neural networks for weather and climate data tasks has shown impressive results across a variety of tasks, e.g., super-resolution (Wang et al, 2021;Yang et al, 2022), temporal modeling (Wang et al, 2018;Stengel et al, 2020), and compression (Dupont et al, 2022;Huang & Hoefler, 2023). Neural network architectures like spherical convolutional neural networks (Cohen et al, 2018) and spherical Fourier neural operators (Bonev et al, 2023) have been tailored for this domain.…”

A Multi-Scale Similarity Based Community Detection for Attributed Complex Networks

“…Coordinate networks [4] (also termed as implicit neural representation or neural fields) are gradually replacing traditional discrete representations in computer vision and graphics. Different from classical matrix-based discrete representation, coordinate networks focus on learning a neural mapping function with low-dimensional coordinates inputs and the corresponding signal values outputs, and have demonstrated the advantages of continuous querying and memory-efficient footprint in various signal representation tasks, such as images [5], [6], [7], scenes [24], [27], [30], [35] and videos [21], [22]. Additionally, coordinate networks could be seamlessly combined with different differentiable physical processes, opening a new way for solving various inverse problems, especially the domain-specific tasks where large-scale labelled datasets are unavailable, such as the shape representation [23], [25], [28], [29], [36], computed tomography reconstruction [26], [31], [32], [33], [34] and inverse rendering for novel view synthesis [37], [38], [41], [42], [100].…”