Hierarchical Cloth Simulation using Deep Neural Networks

Oh, Young Jin; Lee, Tae Min; Lee, In-Kwon

doi:10.1145/3208159.3208162

Cited by 25 publications

(20 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saito et al [SUM14] proposed an upsampling technique that adds physically feasible microscopic detail to coarsened meshes by considering the internal strain at runtime. More recently, Oh et al [OLL18] have shown how to train a deep neural network to upsample low-resolution cloth simulations.…”

Section: Related Workmentioning

confidence: 99%

Learning‐Based Animation of Clothing for Virtual Try‐On

Santesteban

Otaduy

Casas

2019

Computer Graphics Forum

173

225

View full text Add to dashboard Cite

This paper presents a learning‐based clothing animation method for highly efficient virtual try‐on simulation. Given a garment, we preprocess a rich database of physically‐based dressed character simulations, for multiple body shapes and animations. Then, using this database, we train a learning‐based model of cloth drape and wrinkles, as a function of body shape and dynamics. We propose a model that separates global garment fit, due to body shape, from local garment wrinkles, due to both pose dynamics and body shape. We use a recurrent neural network to regress garment wrinkles, and we achieve highly plausible nonlinear effects, in contrast to the blending artifacts suffered by previous methods. At runtime, dynamic virtual try‐on animations are produced in just a few milliseconds for garments with thousands of triangles. We show qualitative and quantitative analysis of results.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning‐Based Animation of Clothing for Virtual Try‐On

Santesteban

Otaduy

Casas

2019

Computer Graphics Forum

173

225

View full text Add to dashboard Cite

show abstract

“…Yan et al takes images as input and uses a neural network to encode the rope state as a set of connected nodes, and apply a bi-directional LSTM to capture the dynamics based on the node representations [10]. For 2D cloth-like objects, a physically-based simulator and fullyconnected networks are combined to perform the simulation [11] and [12] for coarse and fine levels respectively. PlaNet [13] encodes the images by an Autoencoder as a latent code and predicts the future latent representation based on GRU structure.…”

Section: A Deformable Object Dynamics Modelingmentioning

confidence: 99%

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

Weng

Paus

Varava

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates a rich variety of scenarios including different object sizes, object numbers and manipulation actions. We approach dynamics learning by proposing an object-centric graph representation and two modules which are Active Prediction Module (APM) and Position Prediction Module (PPM) based on graph neural networks with an encode-process-decode architecture. At the inference stage, we build a two-stage model based on the learned modules for single time step prediction. We combine modules with different prediction horizons into a mixed-horizon model which addresses long-term prediction. In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks. Supplementary material is available at https://github.com/wengzehang/ deformable_rigid_interaction_prediction

show abstract

“…Chen et al [CYJ ∗ 18] trained a CNN akin to image super resolution to upsample low resolution cloth to high resolution in texture space. Oh et al [OLL18] train fully connected networks (FCNs) for subdividing a triangle into 4 triangles and upsampling the cloth simulation. They average the outputs of the edge vertices predicted by the network on adjacent triangles.…”

Section: Related Workmentioning

confidence: 99%

“…Kavan et al [KGBS11] constrained the low resolution simulation to match the high resolution at low frequencies and trained a linear upsampling operator, optionally with oscillatory modes. During run-time Seiler et al [SSH12] [OLL18] train fully connected networks (FCNs) for subdividing a triangle into 4 triangles and upsampling the cloth simulation. They average the outputs of the edge vertices predicted by the network on adjacent triangles.…”

Section: Related Workmentioning

confidence: 99%

Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network

Chentanez

Macklin

Müller³

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

We introduce a triangle mesh based convolutional neural network. The proposed network structure can be used for problems where input and/or output are defined on a manifold triangle mesh with or without boundary. We demonstrate its applications in cloth upsampling, adding back details to Principal Component Analysis (PCA) compressed cloth, regressing clothing deformation from character poses, and regressing hand skin deformation from bones' joint angles. The data used for training in this work are generated from high resolution extended position based dynamics (XPBD) physics simulations with small time steps and high iteration counts and from an offline FEM simulator, but it can come from other sources. The inference time of our prototype implementation, depending on the mesh resolution and the network size, can provide between 4 to 134 times faster than a GPU based simulator. The inference also only needs to be done for meshes currently visible by the camera.

show abstract

Hierarchical Cloth Simulation using Deep Neural Networks

Cited by 25 publications

References 26 publications

Learning‐Based Animation of Clothing for Virtual Try‐On

Learning‐Based Animation of Clothing for Virtual Try‐On

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network

Contact Info

Product

Resources

About