DML-PL: Deep metric learning based pseudo-labeling framework for class imbalanced semi-supervised learning

Yan, Mi; Hui, Siu Cheung; Li, Ning

doi:10.1016/j.ins.2023.01.074

Cited by 21 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the pseudolabeling with only close data outperformed pseudolabeling with all current data in terms of generalized performance. This suggests that the OD points, which are closely related to the PD in terms of latent clusters, are more informative for propagating learning information . In contrast, including nonclose data points in pseudolabeling may introduce “useless noise”, leading to unnecessary relationships between features and labels and potential overfitting issues.…”

Section: Resultsmentioning

confidence: 99%

Predicting and Evaluating Different Pretreatment Methods on Methane Production from Sludge Anaerobic Digestion via Automated Machine Learning with Ensembled Semisupervised Learning

Cheng,

Xu,

et al. 2023

ACS EST Eng.

View full text Add to dashboard Cite

Accurate prediction of methane production in anaerobic digestion with various pretreatment strategies is of the utmost importance for efficient sludge treatment and resource recovery. Traditional machine learning (ML) algorithms have shown limited prediction accuracy due to challenges in optimizing complex parameters and the scarcity of data. This work proposed a novel integrated system that employed an ensemble semisupervised learning (SSL)-automated ML (AutoML) model with limited variable inputs to reveal the effects of different pretreatments on methane production during sludge digestion with explainable analysis. Considering the direct correlations of the pretreatment type and digestion substrates, the pretreatment type is considered as a hidden variable. Results demonstrated that the AutoML model outperformed the conventional ML models (i.e., support vector regression (SVR), extreme gradient boosting (XGB), etc.), as evidenced by its higher R 2 value. Moreover, the integration of SSL further enhanced the prediction accuracy by effectively leveraging unlabeled data, leading to a reduction in the mean squared error from 11.3 to 9.7. Explainable analysis results revealed the significance of different variables and the utmost importance of operating time, followed by proteins, carbohydrates, chemical oxygen demand, and volatile fatty acids. Furthermore, principal component and correlation analysis unveiled the interconnected relationships among substrate concentration, microbial communities, and metabolic functions for methane production and found that the increasing substrate concentration promoted the enrichment of functional microbial and metabolic functions. These insights shed light on the advantages of SSL-AutoML in predicting methane production in anaerobic digestion systems and elucidate the dependence relationships with key variables, offering valuable guidance for effective sludge pretreatment with enhanced resource recovery capabilities.

show abstract

Section: Resultsmentioning

confidence: 99%

Predicting and Evaluating Different Pretreatment Methods on Methane Production from Sludge Anaerobic Digestion via Automated Machine Learning with Ensembled Semisupervised Learning

Cheng,

Xu,

et al. 2023

ACS EST Eng.

View full text Add to dashboard Cite

show abstract

“…Therefore, we do not filter the OOD samples but (a) learn representation from them and (b) consider soft confidence of being in-domain. Some recent works reveal that utilizing OOD data can be better than filtering them (Luo et al, 2021;Han et al, 2021;Huang et al, 2022). RoPAWS is also in this line of work; however, leveraging OOD data is more natural from the representation learning perspective.…”

Section: Related Workmentioning

confidence: 99%

RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data

Mo¹,

Su²,

Ma³

et al. 2023

Preprint

View full text Add to dashboard Cite

Semi-supervised learning aims to train a model using limited labels. State-of-theart semi-supervised methods for image classification such as PAWS rely on selfsupervised representations learned with large-scale unlabeled but curated data. However, PAWS is often less effective when using real-world unlabeled data that is uncurated, e.g., contains out-of-class data. We propose RoPAWS, a robust extension of PAWS that can work with real-world unlabeled data. We first reinterpret PAWS as a generative classifier that models densities using kernel density estimation. From this probabilistic perspective, we calibrate its prediction based on the densities of labeled and unlabeled data, which leads to a simple closed-form solution from the Bayes' rule. We demonstrate that RoPAWS significantly improves PAWS for uncurated Semi-iNat by +5.3% and curated ImageNet by +0.4%. 1

show abstract

“…Measurable information depends on the tasks handled by a neural network, which can introduce useful metrics for making the input data distinguishable. Among various studies, a pseudolabeling framework for addressing class imbalance was designed [39]. The neural network that forms the framework was trained using the loss function, which calculates the square distances of the embedded vectors of the input data.…”

Section: Metric Learningmentioning

confidence: 99%

Contextual Augmentation Based on Metric-Guided Features for Ocular Axial Length Prediction

Jeong

Han

2023

Mathematics

View full text Add to dashboard Cite

Ocular axial length (AL) measurement is important in ophthalmology because it should be considered prior to operations, such as strabismus surgery or cataract surgery, and the automation of AL measurement with easily obtained retinal fundus images has been studied. However, the performance of deep learning methods inevitably depends on distribution of the data set used, and the lack of data is an issue that needs to be addressed. In this study, we propose a framework for generating pairs of fundus images and their corresponding ALs to improve the AL inference. The generator’s encoder was trained independently using metric learning based on the AL information. A random vector and zero padding were incorporated into the generator to increase data creation flexibility, after which AL information was inserted as conditional information. We verified the effectiveness of this framework by evaluating the performance of AL inference models after training them on a combined data set comprising privately collected actual data and data generated by the proposed method. Compared to using only the actual data set, the mean absolute error and standard deviation of the proposed method decreased from 10.23 and 2.56 to 3.96 and 0.23, respectively, even with a smaller number of layers in the AL prediction models.

show abstract

DML-PL: Deep metric learning based pseudo-labeling framework for class imbalanced semi-supervised learning

Cited by 21 publications

References 30 publications

Predicting and Evaluating Different Pretreatment Methods on Methane Production from Sludge Anaerobic Digestion via Automated Machine Learning with Ensembled Semisupervised Learning

Predicting and Evaluating Different Pretreatment Methods on Methane Production from Sludge Anaerobic Digestion via Automated Machine Learning with Ensembled Semisupervised Learning

RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data

Contextual Augmentation Based on Metric-Guided Features for Ocular Axial Length Prediction

Contact Info

Product

Resources

About