Stability-driven Contact Reconstruction From Monocular Color Images

Joint Hand and Object Pose Estimation from a Single RGB Image using High‐level 2D Constraints

“…We hope that our work can provide new considerations in simultaneous hand‐object reconstruction and inspire work that considers both agreement with the input image, and physical constraints. In the future, we plan to combine our method with other contact information such as physical forces [ZZXW22], and extend the constraints to an object‐agnostic version for more general situations [HVT*19]. In addition, incorporating the proposed 2D constraints into the training of the regression of hand and object pose to enable a more effective method for end‐to‐end hand‐object pose estimation is also a valuable direction.…”

“…Differences in definition of the hand in the optimization step further make comparisons with this method difficult. We believe that, if adapted to the initial pose regression network and physics‐based constraints of SCR [ZZXW22], our method would achieve better results.…”

“…While there is a physical‐based approach, SCR [ZZXW22], which uses quite a different optimization process to our method. However, since its improved results are partly due to the design of its initial pose regression network, which lacks a detailed description.…”

“…[LAZ*22] adopts various dense features to supervise alignment of the hand mesh to the image. [ZZXW22] and [DNMC20] use 2D features to guide regression of final poses of both the hand and the object. While using 2D features, such papers just use them as clues during regression, instead of treating them as hard constraints during final optimization.…”

“…Therefore, many approaches have been applied to obtain a credible result, including distance‐based attraction and repulsion [HVT*19, KYZ*20], learned contact regions [BHHF19, JLWW21, GTT*21], physical simulation [KKB19, KKB20, GHJK20]. Several recent works [GTT*21,YZL*21,CRKM21,ZZXW22] model the contact either by predicting contact [GTT*21, YZL*21, CRKM21] or analyzing physical forces [ZZXW22], and then apply optimization to refine an initial pose estimated by a traditional method.…”