A Strategy for Using Bias and RMSE as Outcomes in Monte Carlo Studies in Statistics

Harwell, Michael R.

doi:10.22237/jmasm/1551907966

Cited by 18 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, some irregularities were observed in which reversals occurred in the results for bias. Previous research has also reported this finding in which different patterns occurred for bias compared to RMSE (Harwell, 2018 ). As Harwell has noted, it may be that average bias is masking important patterns in recovery accuracy compared to RMSE.…”

Section: Discussionsupporting

confidence: 66%

Sample Size Requirements for Applying Diagnostic Classification Models

Şen

Cohen

2021

Front. Psychol.

View full text Add to dashboard Cite

Results of a comprehensive simulation study are reported investigating the effects of sample size, test length, number of attributes and base rate of mastery on item parameter recovery and classification accuracy of four DCMs (i.e., C-RUM, DINA, DINO, and LCDMREDUCED). Effects were evaluated using bias and RMSE computed between true (i.e., generating) parameters and estimated parameters. Effects of simulated factors on attribute assignment were also evaluated using the percentage of classification accuracy. More precise estimates of item parameters were obtained with larger sample size and longer test length. Recovery of item parameters decreased as the number of attributes increased from three to five but base rate of mastery had a varying effect on the item recovery. Item parameter and classification accuracy were higher for DINA and DINO models.

show abstract

Section: Discussionsupporting

confidence: 66%

Sample Size Requirements for Applying Diagnostic Classification Models

Şen

Cohen

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Hence, estimators are chosen not only for their unbiasedness, but also for how widely their estimates vary from sample to sample [21]. RMSE is a metric that commonly used to evaluate bias and estimation accuracy [21,22]. Our result suggested that BGDM estimator is more accurate and has minimum standard error.…”

Section: Discussionmentioning

confidence: 83%

On the Performance of Bayesian Generalized Dissimilarity Model Estimator

Lusiana¹,

Astutik²,

Nurjannah³

et al. 2023

View full text Add to dashboard Cite

“…3. Average relative bias (ARB): the relative difference between the estimated parameter ̂ and its true value , with respect to the true values, over the converged replications N c , expressed as a proportion (Harwell, 2019):…”

Section: Discussionmentioning

confidence: 99%

Identifiability and estimability of Bayesian linear and nonlinear crossed random effects models

Rohloff,

Kohli,

Lock

2024

Brit J Math & Statis

View full text Add to dashboard Cite

Crossed random effects models (CREMs) are particularly useful in longitudinal data applications because they allow researchers to account for the impact of dynamic group membership on individual outcomes. However, no research has determined what data conditions need to be met to sufficiently identify these models, especially the group effects, in a longitudinal context. This is a significant gap in the current literature as future applications to real data may need to consider these conditions to yield accurate and precise model parameter estimates, specifically for the group effects on individual outcomes. Furthermore, there are no existing CREMs that can model intrinsically nonlinear growth. The goals of this study are to develop a Bayesian piecewise CREM to model intrinsically nonlinear growth and evaluate what data conditions are necessary to empirically identify both intrinsically linear and nonlinear longitudinal CREMs. This study includes an applied example that utilizes the piecewise CREM with real data and three simulation studies to assess the data conditions necessary to estimate linear, quadratic, and piecewise CREMs. Results show that the number of repeated measurements collected on groups impacts the ability to recover the group effects. Additionally, functional form complexity impacted data collection requirements for estimating longitudinal CREMs.

show abstract

A Strategy for Using Bias and RMSE as Outcomes in Monte Carlo Studies in Statistics

Cited by 18 publications

References 27 publications

Sample Size Requirements for Applying Diagnostic Classification Models

Sample Size Requirements for Applying Diagnostic Classification Models

On the Performance of Bayesian Generalized Dissimilarity Model Estimator

Identifiability and estimability of Bayesian linear and nonlinear crossed random effects models

Contact Info

Product

Resources

About