Deep Equilibrium Optical Flow Estimation

Bai, Shaojie; Geng, Zhengyang; Savani, Yash; Kolter, J. Zico

doi:10.1109/cvpr52688.2022.00070

Cited by 45 publications

(47 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite this, it is common in the literature for these models to still be called "implicit", perhaps in reference to the fact that the geometry of the scene is defined "implicitly" by the weights of a neural network (a different definition of "implicit" than is used by the SDF literature). Also note that this is a distinct definition of "implicit" than what is commonly used by the deep learning and statistic communities, where "implicit" usually refers to models whose outputs are implicitly defined as fixed points of dynamic systems, and whose gradients are computed using the implicit function theorem [BKK19].…”

Section: Representing Volumesmentioning

confidence: 99%

Advances in Neural Rendering

Tewari

Thies²,

Mildenhall

et al. 2022

Computer Graphics Forum

214

View full text Add to dashboard Cite

show abstract

Section: Representing Volumesmentioning

confidence: 99%

Advances in Neural Rendering

Tewari

Thies²,

Mildenhall

et al. 2022

Computer Graphics Forum

214

View full text Add to dashboard Cite

show abstract

“…In contrast, the naive backpropagation algorithm and ACA are not applicable to implicit functions because they assume explicit ones. (For the backpropagation algorithm for implicit functions, see [58]).…”

Section: B Symplectic Runge-kutta Methods For Adjoint Systemmentioning

confidence: 99%

The Symplectic Adjoint Method: Memory-Efficient Backpropagation of Neural-Network-Based Differential Equations

Matsubara

Miyatake

Yaguchi

2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

The combination of neural networks and numerical integration can provide highly accurate models of continuoustime dynamical systems and probabilistic distributions. However, if a neural network is used n times during numerical integration, the whole computation graph can be considered as a network n times deeper than the original. The backpropagation algorithm consumes memory in proportion to the number of uses times of the network size, causing practical difficulties. This is true even if a checkpointing scheme divides the computation graph into subgraphs. Alternatively, the adjoint method obtains a gradient by a numerical integration backward in time; although this method consumes memory only for single-network use, the computational cost of suppressing numerical errors is high. The symplectic adjoint method proposed in this study, an adjoint method solved by a symplectic integrator, obtains the exact gradient (up to rounding error) with memory proportional to the number of uses plus the network size. The theoretical analysis shows that it consumes much less memory than the naive backpropagation algorithm and checkpointing schemes. The experiments verify the theory, and they also demonstrate that the symplectic adjoint method is faster than the adjoint method and is more robust to rounding errors.

show abstract

“…We train the regularizer h θ by minimizing the discrepancy between a fixed-point x = T θ (x) obtained via (10) and the ground-truth image x using MSE loss…”

Section: Jacobian-free Deep Equilibrium Learningmentioning

confidence: 99%

“…DU is a DL paradigm that has gained popularity due to its ability to systematically connect iterative algorithms and deep neural network architectures (see reviews in [4,26]). DEQ [10] is a related approach that enables training of infinite-depth, weight-tied networks by analytically backpropagating through the fixed points using implicit differentiation. The DEQ output is specified implicitly as a fixed point of an operator T θ parameterized by weights θ…”

Section: Introductionmentioning

confidence: 99%

Deep Equilibrium Learning of Explicit Regularizers for Imaging Inverse Problems

Zou¹,

Liu²,

Wohlberg³

et al. 2023

Preprint

View full text Add to dashboard Cite

There has been significant recent interest in the use of deep learning for regularizing imaging inverse problems. Most work in the area has focused on regularization imposed implicitly by convolutional neural networks (CNNs) pre-trained for image reconstruction. In this work, we follow an alternative line of work based on learning explicit regularization functionals that promote preferred solutions. We develop the Explicit Learned Deep Equilibrium Regularizer (ELDER) method for learning explicit regularizers that minimize a mean-squared error (MSE) metric. ELDER is based on a regularization functional parameterized by a CNN and a deep equilibrium learning (DEQ) method for training the functional to be MSE-optimal at the fixed points of the reconstruction algorithm. The explicit regularizer enables ELDER to directly inherit fundamental convergence results from optimization theory. On the other hand, DEQ training enables ELDER to improve over existing explicit regularizers without prohibitive memory complexity during training. We use ELDER to train several approaches to parameterizing explicit regularizers and test their performance on three distinct imaging inverse problems. Our results show that ELDER can greatly improve the quality of explicit regularizers compared to existing methods, and show that learning explicit regularizers does not compromise performance relative to methods based on implicit regularization.

show abstract

Deep Equilibrium Optical Flow Estimation

Cited by 45 publications

References 27 publications

Advances in Neural Rendering

Advances in Neural Rendering

The Symplectic Adjoint Method: Memory-Efficient Backpropagation of Neural-Network-Based Differential Equations

Deep Equilibrium Learning of Explicit Regularizers for Imaging Inverse Problems

Contact Info

Product

Resources

About