Learning generative texture models with extended Fields-of-Experts

Heess, Nicolas; Williams, Christopher K. I.; Hinton, Geoffrey E.

doi:10.5244/c.23.115

Cited by 48 publications

(30 citation statements)

References 17 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concerning the learned penalty function (d), as it has local minima at two specific points, it prefers specific image structure, implying that it helps to form certain image structure. We also find that this penalty function is exactly the type of bimodal expert functions for texture synthesis employed in [21].…”

Section: Image Denoising Experimentsmentioning

confidence: 65%

See 1 more Smart Citation

On learning optimized reaction diffusion processes for effective image restoration

Chen

Pock

2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

268

304

View full text Add to dashboard Cite

For several decades, image restoration remains an active research topic in low-level computer vision and hence new approaches are constantly emerging. However, many recently proposed algorithms achieve state-of-the-art performance only at the expense of very high computation time, which clearly limits their practical relevance. In this work, we propose a simple but effective approach with both high computational efficiency and high restoration quality. We extend conventional nonlinear reaction diffusion models by several parametrized linear filters as well as several parametrized influence functions. We propose to train the parameters of the filters and the influence functions through a loss based approach. Experiments show that our trained nonlinear reaction diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for image restoration. Due to their structural simplicity, our trained models are highly efficient and are also well-suited for parallel computation on GPUs. AbstractThis is the supplemental material for our CVPR2015 paper entitled "On learning optimized reaction diffusion processes for effective image restoration". In this supplemental material, we give detailed derivations of gradients required in training for corresponding diffusion networks. In addition, we also present more image denoising and JPEG deblocking examples.

show abstract

Section: Image Denoising Experimentsmentioning

confidence: 65%

“…In practice, (21) is computed using a backward manner starting from the last stage. Now the only thing we need to calculate is ∂ut+1 ∂ut .…”

Section: Joint Trainingmentioning

confidence: 99%

On learning optimized reaction diffusion processes for effective image restoration

Chen

Pock

2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

268

304

View full text Add to dashboard Cite

show abstract

Section: Learned Influence Functionsmentioning

confidence: 65%

Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Chen

Pock

2017

IEEE Trans. Pattern Anal. Mach. Intell.

1,193

873

View full text Add to dashboard Cite

Abstract-Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD -Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.

show abstract

“…Therefore the interaction structure is learned as filter coefficients. FoE was extended to bimodal FoE (BiFoE) which uses more informative bimodal potentials, and successfully applied to texture modelling by Heess et al [9]; several state of the art generative texture models have been built on BiFoE, some using various configurations of hidden variables. Kivinen and Williams [19] improved on BiFoE by using gated MRFs [20], and Luo et al [21] investigated convolutional deep belief networks (DBN) and spike-and-slab potential functions.…”

Section: High-order Mgrf Modelsmentioning

confidence: 99%

“…Efros & Leung [42] 0.85 ± 0.03 0.86 ± 0.03 0.86 ± 0.06 0.60 ± 0.08 TmPoT [19] 0.86 ± 0.02 0.87 ± 0.01 0.86 ± 0.02 0.77 ± 0.03 TssRBM [21] 0.89 ± 0.02 0.91 ± 0.01 0.92 ± 0.02 0.76 ± 0.03 2-layer DBN [21] 0.89 ± 0.03 0.91 ± 0.02 0.92 ± 0.03 0.77 ± 0.02 cGRBMs [66] 0.91 ± 0.02 0.93 ± 0.01 0.93 ± 0.01 0.78 ± 0.03 GLD 2 0.66 ± 0.15 0.90 ± 0.04 0.86 ± 0.08 0.78 ± 0.03 Linear filters 0.77 ± 0.02 0.49 ± 0.13 0.89 ± 0.04 0.58 ± 0.06 Combined BP 5 0.80 ± 0.08 0.87 ± 0.06 0.92 ± 0.03 0.81 ± 0.02 Conjoined BP 9 0.77 ± 0.10 0.86 ± 0.06 0.90 ± 0.03 0.80 ± 0.03 Jag-star BP 9 0.84 ± 0.08 0.86 ± 0.07 0.91 ± 0.03 0.80 ± 0.03 Jag-star BP 13 0.73 ± 0.11 0.90 ± 0.05 0.91 ± 0.03 0.79 ± 0.03 used filters as features. A stopping rule for the nesting procedure which is robust to the highly variable quality of CSA samples would also be important if more feature sets are used, to keep the number of nesting iterations low.…”

Section: D21 D53 D77mentioning

confidence: 99%

Texture modelling with nested high-order Markov–Gibbs random fields

Versteegen

Gimel’farb

Riddle

2016

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Currently, Markov-Gibbs random field (MGRF) image models which include high-order interactions are almost always built by modelling responses of a stack of local linear filters. Actual interaction structure is specified implicitly by the filter coefficients. In contrast, we learn an explicit high-order MGRF structure by considering the learning process in terms of general exponential family distributions nested over base models, so that potentials added later can build on previous ones. We relatively rapidly add new features by skipping over the costly optimisation of parameters.We introduce the use of local binary patterns as features in MGRF texture models, and generalise them by learning offsets to the surrounding pixels. These prove effective as high-order features, and are fast to compute. Several schemes for selecting high-order features by composition or search of a small subclass are compared. Additionally we present a simple modification of the maximum likelihood as a texture modelling-specific objective function which aims to improve generalisation by local windowing of statistics.The proposed method was experimentally evaluated by learning high-order MGRF models for a broad selection of complex textures and then performing texture synthesis, and succeeded on much of the continuum from stochastic through irregularly structured to near-regular textures. Learning interaction structure is very beneficial for textures with large-scale structure, although those with complex irregular structure still provide difficulties. The texture models were also quantitatively evaluated on two tasks and found to be competitive with other works: grading of synthesised textures by a panel of observers; and comparison against several recent MGRF models by evaluation on a constrained inpainting task.Keywords: texture synthesis and analysis; high-order MRFs; local binary patterns; structure learning. $ c 2015. Licensed under Creative Commons CC-BY-NC-ND 4.0 http://creativecommons.org/licenses/by-nc-nd/4.0/

show abstract

Learning generative texture models with extended Fields-of-Experts

Cited by 48 publications

References 17 publications

On learning optimized reaction diffusion processes for effective image restoration

On learning optimized reaction diffusion processes for effective image restoration

Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Texture modelling with nested high-order Markov–Gibbs random fields

Contact Info

Product

Resources

About