Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

Yang, Jie; Gao, Lin; Tan, Qingyang; Huang, Yihua; Xia, Shihong; Lai, Yu‐Kun

doi:10.1109/tvcg.2021.3112526

Cited by 11 publications

(7 citation statements)

References 88 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these methods tend to generate complicated character-dependent skeletons, which impacts the retargeting performance when animating a morphologically different character. To ease the restrictions of skeletal rigging, other works [32,33,19] focus on direct mesh deformation by building reliable correspondences among body parts. [32] leverages a bi-harmonic weight deformation framework to produce plausible poses on target mesh when several key points on input meshes are given.…”

Section: Related Workmentioning

confidence: 99%

“…Although these key points on a given source mesh could be automatically identified, this method requires an extra manual process to calibrate the corresponding points on the target mesh. [33] introduce a coarse-to-fine fashion to encode multi-scale mesh deformation into a latent space, and an attention module is leveraged to perceive active regions at different scales. Since the latent spaces are learned from different shapes of the same mesh, retargeting between different characters is impossible with this method.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting

Wang¹,

Huang²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Skeleton-free motion retargeting from an input source motion to the target character. Left: Source motion is animated on a Mixamo [1] character, and the target character is selected in RigNet [31] dataset. The skeletal rigging and skinning on both of the input characters are agnostic during inference. Right: Comparison between SfPT [19] (the state-of-art method) and ours shows the superior performance of our model on handling small-part motions and preserving local motion interdependence. Thereinto, red bubble reflects the consistency of the right hand's direction between the source and target characters. blue bubble reflects local motion independence between the leg and skirt (i.e., whether leg-skirt interpenetration occurs).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting

Wang¹,

Huang²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…With the proliferation of geometric models [3], datadriven deformation [15,16,61] becomes available which analyzes the deformation prior of existing shapes in the dataset and produces more realistic results. At the same time, plenty of data also allows neural networks to be introduced into 3D editing [37,63,73,75]. In addition to the explicit mesh representation, the implicit field can also be edited in combination with a neural network.…”

Section: Related Workmentioning

confidence: 99%

NeRF-Editing: Geometry Editing of Neural Radiance Fields

Yuan¹,

Sun²,

Lai³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Implicit neural rendering, especially Neural Radiance Field (NeRF), has shown great potential in novel view synthesis of a scene. However, current NeRF-based methods cannot enable users to perform user-controlled shape deformation in the scene. While existing works have proposed some approaches to modify the radiance field according to the user's constraints, the modification is limited to color editing or object translation and rotation. In this paper, we propose a method that allows users to perform controllable shape deformation on the implicit representation of the scene, and synthesizes the novel view images of the edited scene without re-training the network. Specifically, we establish a correspondence between the extracted explicit mesh representation and the implicit neural representation of the target scene. Users can first utilize welldeveloped mesh-based deformation methods to deform the mesh representation of the scene. Our method then utilizes user edits from the mesh representation to bend the camera rays by introducing a tetrahedra mesh as a proxy, obtaining the rendering results of the edited scene. Extensive experiments demonstrate that our framework can achieve ideal editing results not only on synthetic data, but also on real scenes captured by users.

show abstract

“…The extracted deformation components can be used to synthesize new shapes. Also for extracting deformation components, Yang et al [246] proposed to use multi-level VAEs, which can extract multi-scale deformation components, achieving better results. Qiao et al [247] proposed bidirectional LSTM with graph convolutions to generate mesh sequences.…”

Section: Editing Via Learning Mesh Deformationmentioning

confidence: 99%

A Revisit of Shape Editing Techniques: From the Geometric to the Neural Viewpoint

Yuan

Lai

et al. 2021

J. Comput. Sci. Technol.

View full text Add to dashboard Cite

3D shape editing is widely used in a range of applications such as movie production, computer games and computer aided design. It is also a popular research topic in computer graphics and computer vision. In past decades, researchers have developed a series of editing methods to make the editing process faster, more robust, and more reliable. Traditionally, the deformed shape is determined by the optimal transformation and weights for an energy formulation. With increasing availability of 3D shapes on the Internet, data-driven methods were proposed to improve the editing results. More recently as the deep neural networks became popular, many deep learning based editing methods have been developed in this field, which are naturally data-driven. We mainly survey recent research studies from the geometric viewpoint to those emerging neural deformation techniques and categorize them into organic shape editing methods and man-made model editing methods. Both traditional methods and recent neural network based methods are reviewed.

show abstract

Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

Cited by 11 publications

References 88 publications

HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting

HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting

NeRF-Editing: Geometry Editing of Neural Radiance Fields

A Revisit of Shape Editing Techniques: From the Geometric to the Neural Viewpoint

Contact Info

Product

Resources

About