MeshMAE: Masked Autoencoders for 3D Mesh Data Analysis

Liang, Yaqian; Zhao, Shanshan; Yu, Baosheng; Zhang, Jing; He, Fazhi

doi:10.1007/978-3-031-20062-5_3

Cited by 18 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The framework attempted to represent optimal features employing both spatial discontinuity specificities, like vertices and neighboring indices. In addition to MeshNet, mesh representation methods have been proposed by several other approaches [18][19][20].…”

Section: D Object Feature Learning In Each Modalitymentioning

confidence: 99%

3D Cross-Modal Retrieval Using Noisy Center Loss and SimSiam for Small Batch Training

2024

KSII TIIS

View full text Add to dashboard Cite

3D Cross-Modal Retrieval (3DCMR) is a task that retrieves 3D objects regardless of modalities, such as images, meshes, and point clouds. One of the most prominent methods used for 3DCMR is the Cross-Modal Center Loss Function (CLF) which applies the conventional center loss strategy for 3D cross-modal search and retrieval. Since CLF is based on center loss, the center features in CLF are also susceptible to subtle changes in hyperparameters and external inferences. For instance, performance degradation is observed when the batch size is too small. Furthermore, the Mean Squared Error (MSE) used in CLF is unable to adapt to changes in batch size and is vulnerable to data variations that occur during actual inference due to the use of simple Euclidean distance between multi-modal features. To address the problems that arise from small batch training, we propose a Noisy Center Loss (NCL) method to estimate the optimal center features. In addition, we apply the simple Siamese representation learning method (SimSiam) during optimal center feature estimation to compare projected features, making the proposed method robust to changes in batch size and variations in data. As a result, the proposed approach demonstrates improved performance in ModelNet40 dataset compared to the conventional methods.

show abstract

Section: D Object Feature Learning In Each Modalitymentioning

confidence: 99%

3D Cross-Modal Retrieval Using Noisy Center Loss and SimSiam for Small Batch Training

2024

KSII TIIS

View full text Add to dashboard Cite

show abstract

“…Most generators using the same type of representation share similar output layers, such as voxel-based works [15,16,28,32], point-cloud-based works [49,147,176], MeshCNN [149] and SubdivNet [151] for mesh representation learning, and OccNet [103] and DeepSDF [102,104] for implicit representations. Very recently, the transformer mechanism has also been employed in 3D deep learning [141,226] with demonstrated advantages compared to classic 3D convolutional networks. The backbone network architecture usually determines the shape representation, which in turn affects the design of the generator.…”

Section: D Backbone Network Designmentioning

confidence: 99%

A survey of deep learning-based 3D shape generation

Yang

2023

Comp. Visual Media

View full text Add to dashboard Cite

Deep learning has been successfully used for tasks in the 2D image domain. Research on 3D computer vision and deep geometry learning has also attracted attention. Considerable achievements have been made regarding feature extraction and discrimination of 3D shapes. Following recent advances in deep generative models such as generative adversarial networks, effective generation of 3D shapes has become an active research topic. Unlike 2D images with a regular grid structure, 3D shapes have various representations, such as voxels, point clouds, meshes, and implicit functions. For deep learning of 3D shapes, shape representation has to be taken into account as there is no unified representation that can cover all tasks well. Factors such as the representativeness of geometry and topology often largely affect the quality of the generated 3D shapes. In this survey, we comprehensively review works on deep-learning-based 3D shape generation by classifying and discussing them in terms of the underlying shape representation and the architecture of the shape generator. The advantages and disadvantages of each class are further analyzed. We also consider the 3D shape datasets commonly used for shape generation. Finally, we present several potential research directions that hopefully can inspire future works on this topic.

show abstract

“…MeshMAE [119] (2022.11) focuses on processing 3D mesh data. The research here is mainly on how to handle meshes and utilize MAE.…”

Section: E 3d and Point Clouds 1) 3d Imagementioning

confidence: 99%

Masked Autoencoders in Computer Vision: A Comprehensive Survey

Zhou,

Liu

2023

IEEE Access

View full text Add to dashboard Cite

Masked autoencoders (MAE) is a deep learning method based on Transformer. Originally used for images, it has now been extended to video, audio, and some other temporal prediction tasks. In the field of computer vision, MAE performs well in classification, prediction, and target detection tasks. In terms of specific application, MAE has made many achievements in medical treatment, geography, 3D point cloud and machine troubleshooting. Since its introduction at the end of 2021, there have been more than 300 related preprints, and MAE has been significantly performed in tier one computer vision conferences during 2022 and 2023. In view of the current popularity of MAE and its future development prospects, we conduct a relatively comprehensive survey of MAE mainly covering officially published articles so far. We comb through and classify the improvements in MAE, demonstrating relatively representative applications in computer vision. Finally, as a summary, we discuss the possible future research directions and development areas based on the characteristics of MAE, hoping our work could be a reference for the future work of MAE.INDEX TERMS Computer vision survey, MAE, masked autoencoders, masked image modeling.

show abstract

MeshMAE: Masked Autoencoders for 3D Mesh Data Analysis

Cited by 18 publications

References 43 publications

3D Cross-Modal Retrieval Using Noisy Center Loss and SimSiam for Small Batch Training

3D Cross-Modal Retrieval Using Noisy Center Loss and SimSiam for Small Batch Training

A survey of deep learning-based 3D shape generation

Masked Autoencoders in Computer Vision: A Comprehensive Survey

Contact Info

Product

Resources

About