Comparison of five imputation methods in handling missing data in a continuous frequency table

Mohammed, M. B.; Zulkafli, Hani Syahida; Adam, Mohd Bakri; Ali, Nada Basheir; Baba, Ishaq Abdullahi

doi:10.1063/5.0053286

Cited by 9 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For missing values, we employed a variety of imputation methods, including mean, median, mode, and K-nearest neighbors (KNN) imputation, selecting the most appropriate technique based on the nature of the data [26]. To identify and correct outliers, we consulted with pharmacy experts to discern whether an anomaly was a true outlier or a data entry error.…”

Section: Data Cleaning and Transformationmentioning

confidence: 99%

Applying Machine Learning and Statistical Forecasting Methods for Enhancing Pharmaceutical Sales Predictions

Fourkiotis,

Tsadiras

2024

Forecasting

View full text Add to dashboard Cite

In today’s evolving global world, the pharmaceutical sector faces an emerging challenge, which is the rapid surge of the global population and the consequent growth in drug production demands. Recognizing this, our study explores the urgent need to strengthen pharmaceutical production capacities, ensuring drugs are allocated and stored strategically to meet diverse regional and demographic needs. Summarizing our key findings, our research focuses on the promising area of drug demand forecasting using artificial intelligence (AI) and machine learning (ML) techniques to enhance predictions in the pharmaceutical field. Supplied with a rich dataset from Kaggle spanning 600,000 sales records from a singular pharmacy, our study embarks on a thorough exploration of univariate time series analysis. Here, we pair conventional analytical tools such as ARIMA with advanced methodologies like LSTM neural networks, all with a singular vision: refining the precision of our sales. Venturing deeper, our data underwent categorisation and were segmented into eight clusters premised on the ATC Anatomical Therapeutic Chemical (ATC) Classification System framework. This segmentation unravels the evident influence of seasonality on drug sales. The analysis not only highlights the effectiveness of machine learning models but also illuminates the remarkable success of XGBoost. This algorithm outperformed traditional models, achieving the lowest MAPE values: 17.89% for M01AB (anti-inflammatory and antirheumatic products, non-steroids, acetic acid derivatives, and related substances), 16.92% for M01AE (anti-inflammatory and antirheumatic products, non-steroids, and propionic acid derivatives), 17.98% for N02BA (analgesics, antipyretics, and anilides), and 16.05% for N02BE (analgesics, antipyretics, pyrazolones, and anilides). XGBoost further demonstrated exceptional precision with the lowest MSE scores: 28.8 for M01AB, 1518.56 for N02BE, and 350.84 for N05C (hypnotics and sedatives). Additionally, the Seasonal Naïve model recorded an MSE of 49.19 for M01AE, while the Single Exponential Smoothing model showed an MSE of 7.19 for N05B. These findings underscore the strengths derived from employing a diverse range of approaches within the forecasting series. In summary, our research accentuates the significance of leveraging machine learning techniques to derive valuable insights for pharmaceutical companies. By applying the power of these methods, companies can optimize their production, storage, distribution, and marketing practices.

show abstract

Section: Data Cleaning and Transformationmentioning

confidence: 99%

Applying Machine Learning and Statistical Forecasting Methods for Enhancing Pharmaceutical Sales Predictions

Fourkiotis,

Tsadiras

2024

Forecasting

View full text Add to dashboard Cite

show abstract

“…Let 𝑦 1 , 𝑦 2 ,…, 𝑦 𝑛0 be the observed values and 𝑦 1 𝑚𝑖𝑠 , 𝑦 2 𝑚𝑖𝑠 ,…, 𝑦 𝑛 𝑝 𝑚𝑖𝑠 be the missing observations of a dataset containing n=𝑛 0 +𝑛 𝑝 observations; the random sample imputation replaces the missing observations with a sample of 𝑛 𝑝 from 𝑛 0 observed values. The random sample estimation has the benefit of working with both continuous and discrete variables (Mohammed et al, 2021).…”

Section: Sample Imputationmentioning

confidence: 99%

Comparison of missing data imputation methods using weather data

Nida

2023

PAKJAS

View full text Add to dashboard Cite

Researchers and data analysts commonly experience with missing data in their field of studies. It is necessary to handle missing data properly to obtain better and more reliable outcomes of any research. The objective of this research is to evaluate different imputation techniques for handling missing observations in weather data. For this purpose weather data of daily rainfall maximum temperature (Tmax) and minimum temperature (Tmin) of 23 stations of Pakistan were taken from Pakistan Metrological department from 1981 to 2020. There are total 14610 observation of each variable and each variable have different number of missing observations at different stations. For the estimation of missing observation at each station mean imputation k nearest neighbors (KNN) imputation predictive mean matching (PMM) imputation and sample imputation techniques are used. Among different imputation techniques the technique which provides minimum root mean square error (RMSE) is selected and used to estimate the missing observations at each station. The result shows that the selection of imputation method is different for each station and mainly depends upon the size of missing observation in the data. For rainfall data the KNN technique is the most appropriate while for Tmax and Tmin the mean imputation technique is recommended for the estimation of missing observation in the data as compared to other methods.

show abstract

“…In the unimodal datasets, participants and features with more than 55% missing values were excluded (deviating from our pre-registered 50% as this would have reduced our sample sizes too much). Remaining missing values were imputed using the k-nearest neighbors 80 method, a relatively simple non-parametric multivariate imputation technique more timeefficient than more complex imputation techniques 81 . Following common practice, we chose k by taking the square root of the number of observations in each of the respective datasets.…”

Section: Missing Value Imputationmentioning

confidence: 99%

Building machine learning prediction models for wellbeing using predictors from the exposome and genome in a population cohort

Pelt,

Habets,

Vinkers

et al. 2023

Preprint

View full text Add to dashboard Cite

Using longitudinal data of a large population cohort (Netherlands Twin Register; collected 1991-2022), we built machine learning prediction models for adult wellbeing on three data modalities (Ns between 702 and 5874). The phenome was captured by parent- and self-reports of psychosocial factors from childhood to adulthood, the genome by polygenic scores, and the exposome by linkage of participants’ postal codes to objective exposures based on registry data. The phenome was found to be highly predictive of wellbeing in an independent test set (R2 = .70), but not the genome (R2 = .01) and exposome (R2 = .01). Adding the genome and exposome to the phenome did not significantly increase prediction. Risk/protective factors such as optimism, personality, social support, and previous wellbeing levels were most predictive. Our findings highlight the importance of monitoring individuals over time for wellbeing prediction and current limitations associated with prediction based on the genome and exposome.

show abstract

Comparison of five imputation methods in handling missing data in a continuous frequency table

Cited by 9 publications

References 27 publications

Applying Machine Learning and Statistical Forecasting Methods for Enhancing Pharmaceutical Sales Predictions

Applying Machine Learning and Statistical Forecasting Methods for Enhancing Pharmaceutical Sales Predictions

Comparison of missing data imputation methods using weather data

Building machine learning prediction models for wellbeing using predictors from the exposome and genome in a population cohort

Contact Info

Product

Resources

About