JFB: Jacobian-Free Backpropagation for Implicit Networks

Fung, Samy Wu; Heaton, Howard; Li, Qiuwei; McKenzie, Daniel; Osher, Stanley; Yin, Wotao

doi:10.1609/aaai.v36i6.20619

Cited by 25 publications

(18 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed FPformer, FPRformer, and FPAformer use fewer parameters and 2 Due to the limited space, the performance of FPRformer is listed in the supplementary materials. 3 Due to the limited space, the comparison results for image JPEG deblocking are listed in the supplementary materials.…”

Section: Discussionmentioning

confidence: 99%

“…the equilibrium point. DEQ and its variants [3] suffer from high computation and inexact gradient to achieve an unacceptable solution. In [34], DEQ is applied in solving inverse problems in imaging, where F(.)…”

Section: A Fixed-point Via Dnnsmentioning

confidence: 99%

“…FPAformer is with T = 10, patch size 120, ks = 1, M = 3. The pre-trained and fine-tuned FPformer, FPRformer, and FPAformer are compared with other methods in SISR (x2, x3, and x4), color and grayscale Gaussian denoising (σ=15, 25,50) and image JPEG deblocking (q=10, 20, 30, 40) 3 . Comparison with DEQ and JFB.…”

Section: Comparison With State-of-the-artmentioning

confidence: 99%

“…We adopted and trained JFB and DEQ framework in [3] for single image superresolution with scale 2. The F θ was built on the network (38.7M parameters) for CIFAR10 in [3]. We finetuned key hyper-parameters, i.e., T and .…”

Section: Comparison With State-of-the-artmentioning

confidence: 99%

“…The popular, exuberant, and efficient Deep Neural Networks (DNNs) techniques provide a DNN-based routine for the Fixed-Point (FP) method to handle optimization problems in computer vision [1]- [3], decision-making [4] and other domains, achieve adorable performance due to existing physical advantages (like GPU computing) or human experiences (like tuning or networks structure settings).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration

Qiao¹,

Liu²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

The great success of Deep Neural Networks (DNNs) has inspired the algorithmic development of DNN-based Fixed-Point (DNN-FP) for computer vision tasks. DNN-FP methods, trained by Back-Propagation Through Time or computing the inaccurate inversion of the Jacobian, suffer from inferior representation ability. Motivated by the representation power of the Transformer, we propose a framework to unroll the FP and approximate each unrolled process via Transformer blocks, called FPformer. To reduce the high consumption of memory and computation, we come up with FPRformer by sharing parameters between the successive blocks. We further design a module to adapt Anderson acceleration to FPRformer to enlarge the unrolled iterations and improve the performance, called FPAformer. In order to fully exploit the capability of the Transformer, we apply the proposed model to image restoration, using self-supervised pre-training and supervised fine-tuning. 161 tasks from 4 categories of image restoration problems are used in the pre-training phase. Hereafter, the pre-trained FPformer, FPRformer, and FPAformer are further fine-tuned for the comparison scenarios. Using self-supervised pre-training and supervised fine-tuning, the proposed FPformer, FPRformer, and FPAformer achieve competitive performance with state-ofthe-art image restoration methods and better training efficiency. FPAformer employs only 29.82% parameters used in SwinIR models, and provides superior performance after fine-tuning. To train these comparison models, it takes only 26.9% time used for training SwinIR models. It provides a promising way to introduce the Transformer in low-level vision tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Fixed-point Via Dnnsmentioning

confidence: 99%

Section: Comparison With State-of-the-artmentioning

confidence: 99%

Section: Comparison With State-of-the-artmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration

Qiao¹,

Liu²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

A survey of contextual optimization methods for decision-making under uncertainty

Sadana,

Chenreddy,

Delage

et al. 2025

European Journal of Operational Research

View full text Add to dashboard Cite

Explainable AI via learning to optimize

Heaton¹,

Fung

2023

Sci Rep

Self Cite

View full text Add to dashboard Cite

Indecipherable black boxes are common in machine learning (ML), but applications increasingly require explainable artificial intelligence (XAI). The core of XAI is to establish transparent and interpretable data-driven algorithms. This work provides concrete tools for XAI in situations where prior knowledge must be encoded and untrustworthy inferences flagged. We use the “learn to optimize” (L2O) methodology wherein each inference solves a data-driven optimization problem. Our L2O models are straightforward to implement, directly encode prior knowledge, and yield theoretical guarantees (e.g. satisfaction of constraints). We also propose use of interpretable certificates to verify whether model inferences are trustworthy. Numerical examples are provided in the applications of dictionary-based signal recovery, CT imaging, and arbitrage trading of cryptoassets. Code and additional documentation can be found at https://xai-l2o.research.typal.academy.

show abstract

JFB: Jacobian-Free Backpropagation for Implicit Networks

Cited by 25 publications

References 41 publications

Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration

Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration

A survey of contextual optimization methods for decision-making under uncertainty

Explainable AI via learning to optimize

Contact Info

Product

Resources

About