Quantized and Regularized Optimization for Coding Images Using Steered Mixtures-of-Experts

Jongebloed, Rolf; Bochinski, Erik; Lange, Lieven; Sikora, Thomas

doi:10.1109/dcc.2019.00044

Cited by 7 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 A further advantage of GD optimization is the ability to apply well-known training techniques explored in neural network based approaches 11 to the SMoE framework, enabling the use of cost functions like mean-squared-error (MSE) or SSIM. Impressive results rivaling JPEG2000 are presented in the work of Jongebloed et al 7 and demonstrate the powerful abilities of the regression model. The basis for the SMoE image model is an edge-aware, continuous non-linear regression function, capable of modeling smooth and sharp transitions in an image without blocking or ringing artifacts, which can be seen in JPEG-like encoders.…”

Section: Introductionmentioning

confidence: 89%

Steered mixture-of-experts autoencoder design for real-time image modelling and denoising

Fleig

Bochinski

Sikora

2023

Real-Time Processing of Image, Depth and Video Information 2023

View full text Add to dashboard Cite

Research in the past years introduced Steered Mixture-of-Experts (SMoE) as a framework to form sparse, edge-aware models for 2D- and higher dimensional pixel data, applicable to compression, denoising, and beyond, and capable to compete with state-of-the-art compression methods. To circumvent the computationally demanding, iterative optimization method used in prior works an autoencoder design is introduced that reduces the run-time drastically while simultaneously improving reconstruction quality for block-based SMoE approaches. Coupling a deep encoder network with a shallow, parameter-free SMoE decoder enforces an efficent and explainable latent representation. Our initial work on the autoencoder design presented a simple model, with limited applicability to compression and beyond. In this paper, we build on the foundation of the first autoencoder design and improve the reconstruction quality by expanding it to models of higher complexity and different block sizes. Furthermore, we improve the noise robustness of the autoencoder for SMoE denoising applications. Our results reveal that the newly adapted autoencoders allow ultra-fast estimation of parameters for complex SMoE models with excellent reconstruction quality, both for noise free input and under severe noise. This enables the SMoE image model framework for a wide range of image processing applications, including compression, noise reduction, and super-resolution.

show abstract

Section: Introductionmentioning

confidence: 89%

Steered mixture-of-experts autoencoder design for real-time image modelling and denoising

Fleig

Bochinski

Sikora

2023

Real-Time Processing of Image, Depth and Video Information 2023

View full text Add to dashboard Cite

show abstract

“…A multitask optimization technique is employed to train the parameters of the SMoE model. 3 As the SSIM metric is supposed to represent the human visual perception more precise than the Mean Squared Error (MSE) the main task is as follows: 10…”

Section: Parameter Optimizationmentioning

confidence: 99%

“…This arrives at a soft partitioning of the input space with arbitrarily-shaped regions in which each expert acts as a regressor. Such regions can be extended over the entire input space such that experts are responsible for thousands of pixels, 3 allowing for efficient representation of the image data. As the SMoE framework is easily scalable towards higher dimensional image modalities, this approach has been extended to video, 4 light field image 5 and light field video processing and coding, 6,7 respectively.…”

Section: Introductionmentioning

confidence: 99%

“…performance of image coding has been massively increased by also including the sparsification approach 10 and introducing the model estimation with already quantized parameters. 3 Presumably, there are still improvements expected for the aforementioned higher dimensional imageries by applying these methods. However, this paper shows that the reconstruction quality suffers from non-linear camera motions within video sequences.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sparse video representation using steered mixture-of-experts with global motion compensation

Jongebloed

Bochinski

Sikora

2023

Real-Time Processing of Image, Depth and Video Information 2023

View full text Add to dashboard Cite

Steered-Mixtures-of-Experts (SMoE) present a unified framework for sparse representation and compression of image data with arbitrary dimensionality. Recent work has shown great improvements in the performance of such models for image and light-field representation. However, for the case of videos the straight-forward application yields limited success as the SMoE framework leads to a piece-wise linear representation of the underlying imagery which is disrupted by nonlinear motion. We incorporate a global motion model into the SMoE framework which allows for higher temporal steering of the kernels. This drastically increases its capabilities to exploit correlations between adjacent frames by only adding 2 to 8 motion parameters per frame to the model but decreasing the required amount of kernels on average by 54.25%, respectively, while maintaining the same reconstruction quality yielding higher compression gains. By halving the number of necessary kernels, we achieve a significant reduction in complexity on the decoder side being a crucial step towards real-time processing.

show abstract

“…Based on MSE, the Gradient Descent optimization for the efficiency of different covariance representations has also been discussed in [13] . It has been proven that the SMoE regressed by SSIM loss can compete with JPEG20 0 0 even for high bitrates [14] . After all, SMoE is a revolutionary approach for image compression that drastically departs from traditional pixel-based coding algorithms.…”

Section: Introductionmentioning

confidence: 99%

Three-dimensional Epanechnikov mixture regression in image coding

et al. 2021

View full text Add to dashboard Cite

Kernel methods have been studied extensively in recent years. We propose a three-dimensional (3-D) Epanechnikov Mixture Regression (EMR) based on our Epanechnikov Kernel (EK) and realize a complete framework for image coding. In our research, we deduce the covariance-matrix form of 3-D Epanechnikov kernels and their correlated statistics to obtain the Epanechnikov mixture models. To apply our theories to image coding, we propose the 3-D EMR which can better model an image in smaller blocks compared with the conventional Gaussian Mixture Regression (GMR). The regressions are all based on our improved Expectation-Maximization (EM) algorithm with mean square error optimization. Finally, we design an Adaptive Mode Selection (AMS) algorithm to realize the best model pattern combination for coding. Our recovered image has clear outlines and superior coding efficiency compared to JPEG below 0.25bpp. Our work realizes an unprecedented theory application by: (1) enriching the theory of Epanechnikov kernel, (2) improving the EM algorithm using MSE optimization, (3) exploiting the EMR and its application in image coding, and (4) AMS optimal modeling combined with Gaussian and Epanechnikov kernel.

show abstract

Quantized and Regularized Optimization for Coding Images Using Steered Mixtures-of-Experts

Cited by 7 publications

References 10 publications

Steered mixture-of-experts autoencoder design for real-time image modelling and denoising

Steered mixture-of-experts autoencoder design for real-time image modelling and denoising

Sparse video representation using steered mixture-of-experts with global motion compensation

Three-dimensional Epanechnikov mixture regression in image coding

Contact Info

Product

Resources

About