Missing data imputation using an iterative denoising autoencoder (IDAE) for dissolved gas analysis

Seo, Boseong; Shin, Jaekyung; Kim, Tae-Jin; Youn, Byeng D.

doi:10.1016/j.epsr.2022.108642

Cited by 12 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Findings suggest that these methods outperform the standard ones and can handle high missing rates of up to 50%. Seo et al [31] tested a denoising autoencoder with kNN pre-imputation for gas data. The comparison was done against common missing value imputation approaches.…”

Section: Approaches Towards Missing Value Imputation Employingmentioning

confidence: 99%

Missing Value Imputation Methods for Electronic Health Records

et al. 2023

View full text Add to dashboard Cite

Electronic health records (EHR) are patient-level information, e.g., laboratory tests and questionnaires, stored in electronic format. Compared to physical records, the EHR alternative allows patients to access their data easily and helps staff with management procedural tasks such as information sharing across different organizations. Moreover, this type of data is commonly used by researchers for predictive and classification purposes, employing statistical and machine learning methods. However, missingness is a phenomenon that is observed very frequently for such measurements. Even though this missingness is often significant, it is usually treated poorly with either case deletion or simple methods, resulting in suboptimal and/or inaccurate predictive results. This happens because the simple methods, e.g., k-nearest neighbors (kNN) and mean/mode imputation, fail in most cases to incorporate the complex relationships that define these medical datasets. To address these limitations, in this paper we test and improve state-of-the-art missing data imputation models and practices. We propose a new missing value imputation method based on denoising autoencoders (DAE) with kNN for the pre-imputation task. We optimize the training methodology by re-applying kNN to the missing data every N epochs using a different value for the variable k each time to yield more accurate results. We also revise a state-of-the-art missing data imputation approach based on a generative adversarial network (GAN). Using this as a baseline, we introduce improvements regarding both the architecture and the training procedure. These models are compared with the ones usually employed within clinical research studies for both the task of imputation and post-imputation prediction. Results show that our proposed deep learning approaches outperform the standard baselines, yielding better imputation and predictive results.

show abstract

Section: Approaches Towards Missing Value Imputation Employingmentioning

confidence: 99%

Missing Value Imputation Methods for Electronic Health Records

et al. 2023

View full text Add to dashboard Cite

show abstract

“…To classify the existence of BT, the DAE model is used. DAE model is the extended edition of AE which aims at recovering the original dataset in noise corrupted dataset [22]. DAE model was depending on the fact that the data preserves its fundamental features, although it is destroyed partially.…”

Section: Bt Classificationmentioning

confidence: 99%

Ensemble Learning Driven Computer-Aided Diagnosis Model for Brain Tumor Classification on Magnetic Resonance Imaging

Vaiyapuri,

Mahalingam,

Ahmad

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Brain tumour (BT) detection involves the process of identifying the presence of a brain tumour in medical imaging, such as MRI scans. BT detection often relies on medical imaging techniques, such as MRI (Magnetic Resonance Imaging), CT (Computed Tomography), or PET (Positron Emission Tomography) scans. Early detection of BT is important and MRI is one of the primary imaging techniques used to diagnose and assess BT. Deep learning (DL) techniques, particularly convolutional neural networks (CNNs) have shown promising results in assisting with BT detection on MRI scans. This study designs an Ensemble Learning Driven Computer-Aided Diagnosis Model for Brain Tumor Classification (ELCAD-BTC) technique on MRIs. The presented system purposes to detect and classify various steps of BTs. The presented system contains a Gabor filtering (GF) approach to remove the noise and increase the quality of MRI images. Moreover, ensemble learning of three DL models namely EfficientNet, DenseNet, and MobileNet is utilized as feature extractors. Furthermore, the denoising autoencoder (DAE) approach can be exploited to detect the presence of BTs. Finally, a social spider optimization algorithm (SSOA) was carried out for the hyperparameter tuning of the DL models. For simulating the improved BT classification outcome, a brief set of simulations occur on BRATS 2015 database.

show abstract

“…Singh et al [25] presented a comprehensive review for fault detection and diagnostics in heating, ventilation, and air conditioning (HVAC) systems, while Dogan and Birant [26] and Zhu et al [23] focused their review on manufacturing applications. Seo et al [27] proposed a methodology for managing missing values of dissolved gas analysis (DGA) data. Wang et al [28] proposed a combined system based on data preprocessing and multi-objective optimization in a wind production application.…”

Section: Related Workmentioning

confidence: 99%

Data Science Application for Failure Data Management and Failure Prediction in the Oil and Gas Industry: A Case Study

et al. 2022

View full text Add to dashboard Cite

In the industrial domain, maintenance is essential to guarantee the correct operations, availability, and efficiency of machinery and systems. With the advent of Industry 4.0, solutions based on machine learning can be used for the prediction of future failures by exploiting historical failure data. Most of the time, these historical data have been collected by companies without a specific structure, schema, or even best practices, resulting in a potential loss of knowledge. In this paper, we analyze the historical data on maintenance alerts of the components of a revamping topping plant (referred to as RT2) belonging to the SARAS group. This analysis is done in collaboration with the ITALTELECO company, a partner of SARAS, that provided the necessary data. The pre-processing methodology to clean and fill these data and extract features useful for a prediction task will be shown. More in detail, we show the process to fill missing fields of these data to provide (i) a category for each fault by using simple natural language processing techniques and performing a clustering, and (ii) a data structure that can enable machine learning models and statistical approaches to perform reliable failure predictions. The data domain in which this methodology is applied is oil and gas, but it may be generalized and reformulated in various industrial and/or academic fields. The ultimate goal of our work is to obtain a procedure that is simple and can be applied to provide strategic support for the definition of an adequate maintenance plan.

show abstract

Missing data imputation using an iterative denoising autoencoder (IDAE) for dissolved gas analysis

Cited by 12 publications

References 24 publications

Missing Value Imputation Methods for Electronic Health Records

Missing Value Imputation Methods for Electronic Health Records

Ensemble Learning Driven Computer-Aided Diagnosis Model for Brain Tumor Classification on Magnetic Resonance Imaging

Data Science Application for Failure Data Management and Failure Prediction in the Oil and Gas Industry: A Case Study

Contact Info

Product

Resources

About