Missing data imputation by the aid of features similarities

Mostafa, Samih M.

doi:10.1504/ijbdm.2020.106883

Cited by 9 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This included addressing missing data points, which can significantly impact model performance. Notably, all missing values represented by "NaN" (Not a Number) were imputed using a suitable method [16,17]. • Data splitting: Following this cleaning process, the dataset was strategically divided into two distinct sets: a training set and a testing set.…”

Section: Methodsmentioning

confidence: 99%

Power PV Forecasting using Machine Learning Algorithms Based on Weather Data in Semi-Arid Climate

Boujoudar,

Bouarfa,

Dadda

et al. 2024

BIO Web Conf.

View full text Add to dashboard Cite

As the energy demand continues to rise, renewable energy sources such as photovoltaic (PV) systems are becoming increasingly popular. PV systems convert solar radiation into electricity, making them an attractive option for reducing reliance on traditional electricity sources and decreasing carbon emissions. To optimize the usage of PV systems, intelligent forecasting algorithms are essential. They enable better decisionmaking regarding cost and energy efficiency, reliability, power optimization, and economic smart grid operations. Machine learning algorithms have proven to be effective in estimating the power of PV systems, improving accuracy by allowing models to understand complex relationships between parameters and evaluate the output power performance of photovoltaic cells. This work presents a study on the use of machine learning algorithms Catboost, LightGBM, XGboost and Random Forest to improve prediction. The study results indicate that using machine learning algorithms LightGBM can improve the accuracy of PV power prediction, which can have significant implications for optimizing energy usage. In addition to reducing uncertainty, machine learning algorithms improve PV systems’ efficiency, reliability, and economic viability, making them more attractive as renewable energy sources.

show abstract

Section: Methodsmentioning

confidence: 99%

Power PV Forecasting using Machine Learning Algorithms Based on Weather Data in Semi-Arid Climate

Boujoudar,

Bouarfa,

Dadda

et al. 2024

BIO Web Conf.

View full text Add to dashboard Cite

show abstract

“…Detecting the missingness mechanism is considered an important step for manipulating MVs. This paper considers and deals with the three kinds of missingness mechanisms [5][6][7][8].…”

Section: Missingness Mechanismsmentioning

confidence: 99%

Handling Missing Values Based on Similarity Classifiers and Fuzzy Entropy Measures

et al. 2022

Self Cite

View full text Add to dashboard Cite

Handling missing values (MVs) and feature selection (FS) are vital preprocessing tasks for many pattern recognition, data mining, and machine learning (ML) applications, involving classification and regression problems. The existence of MVs in data badly affects making decisions. Hence, MVs have to be taken into consideration during preprocessing tasks as a critical problem. To this end, the authors proposed a new algorithm for manipulating MVs using FS. Bayesian ridge regression (BRR) is the most beneficial type of Bayesian regression. BRR estimates a probabilistic model of the regression problem. The proposed algorithm is dubbed as cumulative Bayesian ridge with similarity and Luca’s fuzzy entropy measure (CBRSL). CBRSL reveals how the fuzzy entropy FS used for selecting the candidate feature holding MVs aids in the prediction of the MVs within the selected feature using the Bayesian Ridge technique. CBRSL can be utilized to manipulate MVs within other features in a cumulative order; the filled features are incorporated within the BRR equation in order to predict the MVs for the next selected incomplete feature. An experimental analysis was conducted on four datasets holding MVs generated from three missingness mechanisms to compare CBRSL with state-of-the-art practical imputation methods. The performance was measured in terms of R2 score (determination coefficient), RMSE (root mean square error), and MAE (mean absolute error). Experimental results indicate that the accuracy and execution times differ depending on the amount of MVs, the dataset’s size, and the mechanism type of missingness. In addition, the results show that CBRSL can manipulate MVs generated from any missingness mechanism with a competitive accuracy against the compared methods.

show abstract

“…• Missing completely at random (MCAR) [12]: Assume that the missing value indicator matrix M = M ij and the complete data Y = y ij . The missing data mechanism is described by the conditional distribution of M given Y, say f (M Y, ∅) where ∅ represents the unknown parameter.…”

Section: Missingness Mechanismsmentioning

confidence: 99%

“…• Missing at random (MAR) [12]: Let Y mis and Y obs denote missing data and observed data, respectively. If the Missingness do not depend on the data that are missing, but depends only on Y obs of Y, then, f (M|Y, ∅) = f (M|Y obs , ∅) f or all Y mis , ∅…”

Section: Missingness Mechanismsmentioning

confidence: 99%

“…Firstly, the covariance matrix and the mean are calculated from the available information within a dataset, then the missing values are manipulated by the use of the covariance matrix and the mean [13]. Manipulating missing data using fuzzy modeling after a statistical classifier has improved the accuracy of missing values imputation in intensive care units (ICUs) database [12]. Manipulating missing data using some popular methods: complete case analysis, K-nearest neighbor, mean imputation, and median imputation were studied against each other, the comparison was studied on 12 datasets [18].…”

Section: Relevant Imputation Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

CBRL and CBRC: Novel Algorithms for Improving Missing Value Imputation Accuracy Based on Bayesian Ridge Regression

et al. 2020

View full text Add to dashboard Cite

In most scientific studies such as data analysis, the existence of missing data is a critical problem, and selecting the appropriate approach to deal with missing data is a challenge. In this paper, the authors perform a fair comparative study of some practical imputation methods used for handling missing values against two proposed imputation algorithms. The proposed algorithms depend on the Bayesian Ridge technique under two different feature selection conditions. The proposed algorithms differ from the existing approaches in that they cumulate the imputed features; those imputed features will be incorporated within the Bayesian Ridge equation for predicting the missing values in the next incomplete selected feature. The authors applied the proposed algorithms on eight datasets with different amount of missing values created from different missingness mechanisms. The performance was measured in terms of imputation time, root-mean-square error (RMSE), coefficient of determination (R2), and mean absolute error (MAE). The results showed that the performance varies depending on missing values percentage, size of the dataset, and the missingness mechanism. In addition, the performance of the proposed methods is slightly better.

show abstract

Missing data imputation by the aid of features similarities

Cited by 9 publications

References 24 publications

Power PV Forecasting using Machine Learning Algorithms Based on Weather Data in Semi-Arid Climate

Power PV Forecasting using Machine Learning Algorithms Based on Weather Data in Semi-Arid Climate

Handling Missing Values Based on Similarity Classifiers and Fuzzy Entropy Measures

CBRL and CBRC: Novel Algorithms for Improving Missing Value Imputation Accuracy Based on Bayesian Ridge Regression

Contact Info

Product

Resources

About