Multi-View 3D Shape Recognition via Correspondence-Aware Deep Learning

Xu, Yong; Zheng, Chaoda; Xu, Ruotao; Quan, Yuhui; Ling, Haibin

doi:10.1109/tip.2021.3082310

Cited by 31 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Simultaneously, we anticipate that the diagnosis results of the AI system can be quantitatively described in the original images using Grad-CAM 29 , alleviating the black box critique of deep neural networks. For the deep-learning-based CT-Nets, two modified 3D-ResNet-18 networks 30 (Supplementary Fig. 3 ) were designed for the identification of radiological abnormalities and the diagnosis of major respiratory diseases in 3D CT volumes.…”

Section: Resultsmentioning

confidence: 99%

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Wang

Ma²,

Zhang³

et al. 2022

npj Digit. Med.

View full text Add to dashboard Cite

Respiratory diseases impose a tremendous global health burden on large patient populations. In this study, we aimed to develop DeepMRDTR, a deep learning-based medical image interpretation system for the diagnosis of major respiratory diseases based on the automated identification of a wide range of radiological abnormalities through computed tomography (CT) and chest X-ray (CXR) from real-world, large-scale datasets. DeepMRDTR comprises four networks (two CT-Nets and two CXR-Nets) that exploit contrastive learning to generate pre-training parameters that are fine-tuned on the retrospective dataset collected from a single institution. The performance of DeepMRDTR was evaluated for abnormality identification and disease diagnosis on data from two different institutions: one was an internal testing dataset from the same institution as the training data and the second was collected from an external institution to evaluate the model generalizability and robustness to an unrelated population dataset. In such a difficult multi-class diagnosis task, our system achieved the average area under the receiver operating characteristic curve (AUC) of 0.856 (95% confidence interval (CI):0.843–0.868) and 0.841 (95%CI:0.832–0.887) for abnormality identification, and 0.900 (95%CI:0.872–0.958) and 0.866 (95%CI:0.832–0.887) for major respiratory diseases’ diagnosis on CT and CXR datasets, respectively. Furthermore, to achieve a clinically actionable diagnosis, we deployed a preliminary version of DeepMRDTR into the clinical workflow, which was performed on par with senior experts in disease diagnosis, with an AUC of 0.890 and a Cohen’s k of 0.746–0.877 at a reasonable timescale; these findings demonstrate the potential to accelerate the medical workflow to facilitate early diagnosis as a triage tool for respiratory diseases which supports improved clinical diagnoses and decision-making.

show abstract

Section: Resultsmentioning

confidence: 99%

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Wang

Ma²,

Zhang³

et al. 2022

npj Digit. Med.

View full text Add to dashboard Cite

show abstract

“…We are well aware that with the increase of computing power and the number of shape datasets available, deep learning techniques dominate the domain of shape comparisons (see, for example, [49][50][51][52]). Applications of deep learning, however, are contingent on the access to large datasets of shapes that are relevant to the shapes under study.…”

Section: (A) (B) (C)mentioning

confidence: 99%

A Physicist’s View on Partial 3D Shape Matching

Koehl

Orland

2023

Algorithms

View full text Add to dashboard Cite

A new algorithm is presented to compute nonrigid, possibly partial comparisons of shapes defined by unstructured triangulations of their surfaces. The algorithm takes as input a pair of surfaces with each surface given by a distinct and unrelated triangulation. Its goal is to define a possibly partial correspondence between the vertices of the two triangulations, with a cost associated with this correspondence that can serve as a measure of the similarity of the two shapes. To find this correspondence, the vertices in each triangulation are characterized by a signature vector of features. We tested both the LD-SIFT signatures, based on the concept of spin images, and the wave kernel signatures obtained by solving the Shrödinger equation on the triangulation. A cost matrix C is constructed such that C(k,l) is the norm of the difference of the signature vectors of vertices k and l. The correspondence between the triangulations is then computed as the transport plan that solves the optimal transport or optimal partial transport problem between their sets of vertices. We use a statistical physics approach to solve these problems. The presentation of the proposed algorithm is complemented with examples that illustrate its effectiveness and manageable computing cost.

show abstract

“…Model-based 3D model retrieval methods [5][6][7] put emphasis on learning discriminative 3D features with certain 3D forms, such as point cloud, 5 voxel, 8 and multi-view 9 representations. For example, PVRNet 6 well fuses the multi-view features and point cloud features based on a relation score module to obtain a unified representation for 3D shapes.…”

Section: Introductionmentioning

confidence: 99%

Source‐enhanced prototypical alignment for single image 3D model retrieval

Song

Wang

Zhang

et al. 2022

Computer Animation & Virtual

View full text Add to dashboard Cite

Single image 3D model retrieval has attracted a lot of attentions with the convenience of organizing large-scale unlabeled 3D models. Existing methods transfer the knowledge from well-annotated 2D images (i.e., source domain) to unlabeled 3D models (i.e., target domain) to improve the discriminability of 3D models and align the feature distributions of 2D images and 3D models. However, during the alignment, the feature learning target of improving the discriminability of 3D models sometimes confuses the boundaries between 2D image categories, where prior methods ignore keeping the discriminability of 2D images. Motivated by this observation, we propose a source-enhanced prototypical alignment framework to first remain the discriminability of 2D images and then guide the category-level cross-domain alignment with better image representations. Specifically, a novel separation and compactness loss is proposed for images to separate the samples from different categories and compact the samples within the same category. Then we perform prototypical alignment to make 2D image features assist in the discriminative feature learning for 3D models. We evaluate the proposed method on the commonly used cross-domain 3D model retrieval benchmarks, namely MI3DOR and MI3DOR-2, and the results demonstrate the effectiveness of the proposed method.

show abstract

Multi-View 3D Shape Recognition via Correspondence-Aware Deep Learning

Cited by 31 publications

References 42 publications

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

A Physicist’s View on Partial 3D Shape Matching

Source‐enhanced prototypical alignment for single image 3D model retrieval

Contact Info

Product

Resources

About