Tangent Space Backpropagation for 3D Transformation Groups

Teed, Zachary; Deng, Jia

doi:10.1109/cvpr46437.2021.01020

Cited by 30 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both the Fuzzy Metaballs and PyTorch3D optimization use an axis-angle, 3 parameter rotation estimation. These is some evidence suggesting PyTorch Autograd for SO(3) might be unstable at times in its native form [69]. Lastly, we suspect that the same scale invariance issues we address in Section 7.1 may exist for the PyTorch3D baselines.…”

Section: E Softrasterizer Performancementioning

confidence: 74%

Approximate Differentiable Rendering with Algebraic Surfaces

Keselman¹,

Hebert²

2022

Preprint

View full text Add to dashboard Cite

Differentiable renderers provide a direct mathematical link between an object's 3D representation and images of that object. In this work, we develop an approximate differentiable renderer for a compact, interpretable representation, which we call Fuzzy Metaballs. Our approximate renderer focuses on rendering shapes via depth maps and silhouettes. It sacrifices fidelity for utility, producing fast runtimes and high-quality gradient information that can be used to solve vision tasks. Compared to mesh-based differentiable renderers, our method has forward passes that are 5x faster and backwards passes that are 30x faster. The depth maps and silhouette images generated by our method are smooth and defined everywhere. In our evaluation of differentiable renderers for pose estimation, we show that our method is the only one comparable to classic techniques. In shape from silhouette, our method performs well using only gradient descent and a per-pixel loss, without any surrogate losses or regularization. These reconstructions work well even on natural video sequences with segmentation artifacts. Project page: https://leonidk.github.io/fuzzy-metaballs

show abstract

Section: E Softrasterizer Performancementioning

confidence: 74%

Approximate Differentiable Rendering with Algebraic Surfaces

Keselman¹,

Hebert²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It is based mainly on three works: The quaternion-based accurate pose representation for multi-source spatial data, the fast and accurate 2D/3D spatial relationship calculation of spatial objects and the high-speed asynchronous method for vector visualization. In the camera's quaternion-based pose transformation, any increment (adjacent pose transformation relation) is calculated on the tangent space SE(3) [98] at the identity matrix, and the obtained increment is exponentially mapped back to the global spatial pose of the moving AR device. This smoothed difference property of quaternions avoids singularities and ensures that small transformation matrices can also be represented, supporting smooth expression of differences between arbitrary directions.…”

Section: Discussionmentioning

confidence: 99%

An Accurate and Efficient Quaternion-Based Visualization Approach to 2D/3D Vector Data for the Mobile Augmented Reality Map

Wang

Huang

Shi

2022

IJGI

View full text Add to dashboard Cite

Increasingly complex vector map applications and growing multi-source spatial data pose a serious challenge to the accuracy and efficiency of vector map visualization. It is true especially for real-time and dynamic scene visualization in mobile augmented reality, with the dramatic development of spatial data sensing and the emergence of AR-GIS. Such issues can be decomposed into three issues: accurate pose representation, fast and precise topological relationships computation and high-performance acceleration methods. To solve these issues, a novel quaternion-based real-time vector map visualization approach is proposed in this paper. It focuses on precise position and orientation representation, accurate and efficient spatial relationships calculation and acceleration parallel rendering in mobile AR. First, a quaternion-based pose processing method for multi-source spatial data is developed. Then, the complex processing of spatial relationships is mapped into simple and efficient quaternion-based operations. With these mapping methods, spatial relationship operations with large computational volumes can be converted into efficient quaternion calculations, and then the results are returned to respond to the interaction. Finally, an asynchronous rendering acceleration mechanism is also presented in this paper. Experiments demonstrated that the method proposed in this paper can significantly improve vector visualization of the AR map. The new approach, when compared to conventional visualization methods, provides more stable and accurate rendering results, especially when the AR map has strenuous movements and high frequency variations. The smoothness of the user interaction experience is also significantly improved.

show abstract

“…Lie groups are widely used in robotics and vision to represent 2D/3D positions and rotations [32]. Due to their non-Euclidean geometry, it is difficult to apply them to deep learning, which primarily operates with Euclidean tensors, but recently there is growing interest in making them compatible [23,[52][53][54][55][56]. LieTorch [53] generalizes automatic differentiation on the Lie group tangent space through local parameterization around the identity, but the implementation is complex since every operation requires a custom kernel.…”

Section: Application Agnostic Interfacementioning

confidence: 99%

“…Due to their non-Euclidean geometry, it is difficult to apply them to deep learning, which primarily operates with Euclidean tensors, but recently there is growing interest in making them compatible [23,[52][53][54][55][56]. LieTorch [53] generalizes automatic differentiation on the Lie group tangent space through local parameterization around the identity, but the implementation is complex since every operation requires a custom kernel. In contrast, Theseus computes common Lie group operators, e.g., the exponential and logarithm map, inverse, composition, etc., in closed form, and provides their corresponding analytical derivatives on the tangent space.…”

Section: Application Agnostic Interfacementioning

confidence: 99%

Theseus: A Library for Differentiable Nonlinear Optimization

Villaseñor-Pineda¹,

Fan²,

Monge³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framework for end-to-end structured learning in robotics and vision. Existing DNLS implementations are application specific and do not always incorporate many ingredients important for efficiency. Theseus is application-agnostic, as we illustrate with several example applications that are built using the same underlying differentiable components, such as second-order optimizers, standard costs functions, and Lie groups. For efficiency, Theseus incorporates support for sparse solvers, automatic vectorization, batching, GPU acceleration, and gradient computation with implicit differentiation and direct loss minimization. We do extensive performance evaluation in a set of applications, demonstrating significant efficiency gains and better scalability when these features are incorporated. Project page: https://sites.google.com/view/theseus-ai/ IntroductionReconciling traditional approaches with deep learning to leverage their complementary strengths is a common thread in a large body of recent work in robotics. In particular, an emerging trend is to differentiate through nonlinear least squares [1] which is a second-order optimization formulation at the heart of many problems in robotics [2-7] and vision [8][9][10][11][12][13]. Optimization layers as inductive priors in neural models have been explored in machine learning with convex optimization [14,15] and in meta learning with gradient descent [16,17] based first-order optimization.Differentiable nonlinear least squares provides a general scheme to encode inductive priors, as the objective function can be partly parameterized by neural models and partly with engineered domain-specific differentiable models. Here, input tensors define a sum of weighted squares objective function and output tensors are minima of that objective. In contrast, typical neural layers take input tensors through a linear transformation and some element-wise nonlinear activation function.The ability to compute gradients end-to-end is retained by differentiating through the optimizer which allows neural models to train on the final task loss, while also taking advantage of priors captured by the optimizer. The flexibility of such a scheme has led to promising state-of-the-art results in a wide range of applications such as structure from motion [18], motion planning [19], SLAM [20,21], bundle adjustment [22], state estimation [23,24], image alignment [25] with other applications like manipulation and tactile sensing [26,27], control [28], human pose tracking [29,30] to be explored. However, existing implementations from above are application specific, common underlying tools like optimizers get reimplemented, and features like sparse solvers, batching, and GPU support that impact efficiency are not always included. This has led to a fragmented literature where it is difficult to start work on new ideas or to build on the...

show abstract

Tangent Space Backpropagation for 3D Transformation Groups

Cited by 30 publications

References 26 publications

Approximate Differentiable Rendering with Algebraic Surfaces

Approximate Differentiable Rendering with Algebraic Surfaces

An Accurate and Efficient Quaternion-Based Visualization Approach to 2D/3D Vector Data for the Mobile Augmented Reality Map

Theseus: A Library for Differentiable Nonlinear Optimization

Contact Info

Product

Resources

About