Evaluating Performance of Missing Data Imputation Methods in IRT Analyses

Kalkan, Ömür Kaya; Kara, Yusuf; Kelecioğlu, Hülya

doi:10.21449/ijate.430720

Cited by 20 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since differences between missing data rate are not important, Random Forest and MICE can support up to 50% of missing rate at an optimal sample size of 200. Results are not in agreement with those of [43] who found that error decreased when the sample size increase no matter the missing rate. The difference might be explained by the fact that in our study imputed data pass through the network before evaluating the performances.…”

Section: Effect Of Imputation Methods On Size and Missing Data Ratecontrasting

confidence: 99%

Robustness of Imputation Methods with Backpropagation Algorithm in Nonlinear Multiple Regression

Hounmenou¹,

Behingan²,

Chrysostome³

et al. 2021

Preprint

View full text Add to dashboard Cite

Missing observations constitute one of the most important issues in data analysis in applied research studies. The magnitude and their structure impact parameters estimation in the modeling with important consequences for decision-making. This study aims to evaluate the efficiency of imputation methods combined with the backpropagation algorithm in a nonlinear regression context. The evaluation is conducted through a simulation study including sample sizes (50, 100, 200, 300 and 400) with different missing data rates (10, 20, 30 40 and 50%) and three missingness mechanisms (MCAR, MAR and MNAR). Four imputation methods (Last Observation Carried Forward, Random Forest, Amelia and MICE) were used to impute datasets before making prediction with backpropagation. 3-MLP model was used by varying the activation functions (Logistic-Linear, Logistic-Exponential, TanH-Linear and TanH-Exponentiel), the number of nodes in the hidden layer (3 - 15) and the learning rate (20 - 70%). Analysis of the performance criteria (R2, r and RMSE) of the network revealed good performances when it is trained with TanH-Linear functions, 11 nodes in the hidden layer and a learning rate of 50%. MICE and Random Forest were the most appropriate for data imputation. These methods can support up to 50% of missing rate with an optimal sample size of 200.

show abstract

Section: Effect Of Imputation Methods On Size and Missing Data Ratecontrasting

confidence: 99%

Robustness of Imputation Methods with Backpropagation Algorithm in Nonlinear Multiple Regression

Hounmenou¹,

Behingan²,

Chrysostome³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The use of EM as an imputation method has been studied in the literature of psychometrics (e.g., [47,52,53]). Results suggested that the EM imputation might work well for polytomous data (e.g., [53]), whereas it yielded a greater bias in IRT parameter estimations than other methods (e.g., MI, [47,52]) when the data were dichotomous. Part of the reason, according to Finch [47], might lie in that the multivariate normality assumption usually does not hold for dichotomous data.…”

Section: Em Imputationmentioning

confidence: 99%

Handling Missing Responses in Psychometrics: Methods and Software

Dai

2021

Psych

View full text Add to dashboard Cite

The presence of missing responses in assessment settings is inevitable and may yield biased parameter estimates in psychometric modeling if ignored or handled improperly. Many methods have been proposed to handle missing responses in assessment data that are often dichotomous or polytomous. Their applications remain nominal, however, partly due to that (1) there is no sufficient support in the literature for an optimal method; (2) many practitioners and researchers are not familiar with these methods; and (3) these methods are usually not employed by psychometric software and missing responses need to be handled separately. This article introduces and reviews the commonly used missing response handling methods in psychometrics, along with the literature that examines and compares the performance of these methods. Further, the use of the TestDataImputation package in R is introduced and illustrated with an example data set and a simulation study. Corresponding R codes are provided.

show abstract

“…Database schema differences among the contributing sources will cause data sets originated from the source with lack of attributes to be missing values within the merged data sets [2]. • Respondents' answering behavior: In survey data, missing values are often caused by reasons like respondents refuse to answer the survey or they do not understand the questions in the questionnaires [25], [26]. • Error in data collection tools: Research data is also prone to missing values problem due to an error in data collection tool (such as sensors) or human researcher's fault.…”

Section: A Reasons Of Missing Datamentioning

confidence: 99%

“…This approach leads to the lack of sufficient data to complete an analysis and thus may give misleading results [27], [28]. Kalkan (2018) stated that although there is a direct correlation between the rate of missing values and the quality of statistical analysis, there is no acceptable proportion of missing values in the data set for the correct statistical conclusion [29], [26]. However, Schafer (1999) argued that the ratio of 5% or less of missing values is inconsequential [30].…”

Section: A Reasons Of Missing Datamentioning

confidence: 99%

An Assessment of Open Data Sets Completeness

Ali¹,

Emran²,

Asmai³

et al. 2019

IJACSA

View full text Add to dashboard Cite

The rapid growth of open data sources is driven by free-of-charge contents and ease of accessibility. While it is convenient for public data consumers to use data sets extracted from open data sources, the decision to use these data sets should be based on data sets' quality. Several data quality dimensions such as completeness, accuracy, and timeliness are common requirements to make data fit for use. More importantly, in many cases, high-quality data sets are desirable in ensuring reliable outcomes of reports and analytics. Even though many open data sources provide data quality guidelines, the responsibility to ensure data of high quality requires commitment from data contributors. In this paper, an initial investigation on the quality of open data sets in terms of completeness dimension was conducted. In particular, the results of the missing values in 20 open data sets measurement were extracted from the open data sources. The analysis covered all the missing values representations which are not limited to nulls or blank spaces. The results exhibited a range of missing values ratios that indicated the level of the data sets completeness. The limited coverage of this analysis does not hinder understanding of the current level of data completeness of open data sets. The findings may motivate open data providers to design initiatives that will empower data quality policy and guidelines for data contributors. In addition, this analysis may assist public data users to decide on the acceptability of open data sets by applying the simple methods proposed in this paper or performing data cleaning actions to improve the completeness of the data sets concerned.

show abstract

Evaluating Performance of Missing Data Imputation Methods in IRT Analyses

Cited by 20 publications

References 32 publications

Robustness of Imputation Methods with Backpropagation Algorithm in Nonlinear Multiple Regression

Robustness of Imputation Methods with Backpropagation Algorithm in Nonlinear Multiple Regression

Handling Missing Responses in Psychometrics: Methods and Software

An Assessment of Open Data Sets Completeness

Contact Info

Product

Resources

About