Twin neural network regression is a semi-supervised regression algorithm

Wetzel, Sebastian Johann; Melko, Roger G.; Tamblyn, Isaac

doi:10.1088/2632-2153/ac9885

Cited by 4 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-supervised regression COREG [22] Uses two models to generate pseudo-labels from unlabeled data SSDKL [33] Combines neural networks and kernel methods to minimize prediction variance of unlabeled data TNNR [23] Trains the difference between predictions of two unlabeled data pairs through loop consistency UCVME [21] Improves the quality of uncertainty estimates on unlabeled data using a…”

Section: Category Methods Descriptionmentioning

confidence: 99%

“…However, COREG comes with increased computational costs because of the separate training of the two K-nearest neighbors regression models and reduced efficiency caused by information transfer between the models. Wetzel et al, (2022) [23] proposed twin neural network regression (TNNR) that trains the difference between the predictions of two unlabeled data pairs through loop consistency. Although TNNR enables learning from unlabeled data through loop consistency, it requires two or more independent models, increasing computational costs.…”

Section: Related Work a Semi-supervised Learning Methodsmentioning

confidence: 99%

“…SSL enables the deep learning models to leverage unlabeled data to obtain additional information, thereby enhancing their prediction abilities [20]. Previous studies on semi-supervised regression (SSR) involved training multiple models to generate highquality pseudo-labels for unlabeled data [21][22][23]. More recently, the ensemble Bayesian neural network (BNN) method was introduced to adjust for pseudo-label quality of unlabeled data by estimating uncertainty associated with unlabeled data [21].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction

Sim,

Bae,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

The rapid advancement of artificial intelligence has observed increased application in predicting vehicle interior noise levels within the automotive industry. However, the collection of labeled data for training models in this context involves significant costs. Previous studies in semi-supervised regression (SSR) have effectively mitigated the reliance on labeled data by incorporating unlabeled data. Nonetheless, these approaches often introduce a high computational cost due to the training of multiple models and data sampling. This study introduces SpecRegMatch, a novel SSR method aimed at addressing the computational cost associated with training by leveraging a single model, thus eliminating the need for multiple data samplings. SpecRegMatch integrates consistency regularization and information maximization to robustly train the model, achieved through various augmentations applied to both the embedding vectors and predicted values. Experimental results demonstrate that SpecRegMatch achieves state-of-the-art performance across various scenarios, even when using a single model. It attains a remarkable performance, as indicated by an 𝑅 2 score of 0.434. This is especially noteworthy in scenarios where labeled data is scarce. You can access the code for our proposed method at https://github.com/sejin-sim/SpecRegMatch.

show abstract

Section: Category Methods Descriptionmentioning

confidence: 99%

Section: Related Work a Semi-supervised Learning Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction

Sim,

Bae,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

ConfRank: Improving GFN-FF Conformer Ranking with Pairwise Training

Hölzer,

Oerder,

Grimme

et al. 2024

J. Chem. Inf. Model.

View full text Add to dashboard Cite

Conformer ranking is a crucial task for drug discovery, with methods for generating conformers often based on molecular (meta)dynamics or sophisticated sampling techniques. These methods are constrained by the underlying force computation regarding runtime and energy ranking accuracy, limiting their effectiveness for large-scale screening applications. To address these ranking limitations, we introduce ConfRank, a machine learning-based approach that enhances conformer ranking using pairwise training. We demonstrate its performance using GFN-FF-generated conformer ensembles, leveraging the DimeNet ++ architecture trained on pairs of 159 760 uncharged organic compounds from the GEOM data set with r 2 SCAN-3c reference level. Instead of predicting only on single molecules, this approach captures relative energy differences between conformers, leading to a significant improvement of the overall conformational ranking, outperforming GFN-FF and GFN2-xTB. Thereby, the pairwise RMSD of the relative energy difference of two conformers can be reduced from 5.65 to 0.71 kcal mol −1 on the test data set, allowing to correctly identify up to 81% of all lowest lying conformers correctly (GFN-FF: 10%, GFN2-xTB: 47%). The ConfRank approach is cost-effective, allowing for scalable deployment on both CPU and GPU, achieving runtime accelerations by up to 2 orders of magnitude compared to GFN2-xTB. Out-of-sample investigations on CREST-generated conformer ensembles from the QM9 data set and conformers taken from an extended GMTKN55 data set show promising results for the robustness of this approach. Thereby, ranking correlation coefficient such as Spearman can be improved to 0.90 (GFN-FF: 0.39, GFN2-xTB: 0.84) reducing the probability of an incorrect sign flip in pairwise energy comparison from 32 to 7%. On the extended GMTKN55 subsets the pairwise MAD (RMSD) could be reduced on almost all subsets by up to 62% (58%) with an average improvement of 30% (29%). Moreover, an exemplary case study on vancomycin shows similar performance, indicating applicability to larger (bio)molecular structures. Furthermore, we motivate the usage of the pairwise training approach from a theoretical perspective, highlighting that while pairwise training can lead to a decline in single sample prediction of absolute energies for ML models, it significantly enhances conformer ranking performance. The data and models used in this study are available at https://github.com/grimme-lab/confrank.

show abstract

Beyond independent error assumptions in large GNN atomistic models

Janghoon

Tian

Kitchin

et al. 2023

The Journal of Chemical Physics

View full text Add to dashboard Cite

The calculation of relative energy difference has significant practical applications, such as determining adsorption energy, screening for optimal catalysts with volcano plots, and calculating reaction energies. Although Density Functional Theory (DFT) is effective in calculating relative energies through systematic error cancellation, the accuracy of Graph Neural Networks (GNNs) in this regard remains uncertain. To address this, we analyzed ∼483 × 106 pairs of energy differences predicted by DFT and GNNs using the Open Catalyst 2020-Dense dataset. Our analysis revealed that GNNs exhibit a correlated error that can be reduced through subtraction, challenging the assumption of independent errors in GNN predictions and leading to more precise energy difference predictions. To assess the magnitude of error cancellation in chemically similar pairs, we introduced a new metric, the subgroup error cancellation ratio. Our findings suggest that state-of-the-art GNN models can achieve error reduction of up to 77% in these subgroups, which is comparable to the error cancellation observed with DFT. This significant error cancellation allows GNNs to achieve higher accuracy than individual energy predictions and distinguish subtle energy differences. We propose the marginal correct sign ratio as a metric to evaluate this performance. Additionally, our results show that the similarity in local embeddings is related to the magnitude of error cancellation, indicating the need for a proper training method that can augment the embedding similarity for chemically similar adsorbate–catalyst systems.

show abstract

Twin neural network regression is a semi-supervised regression algorithm

Cited by 4 publications

References 15 publications

Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction

Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction

ConfRank: Improving GFN-FF Conformer Ranking with Pairwise Training

Beyond independent error assumptions in large GNN atomistic models

Contact Info

Product

Resources

About