A Method to Increase the Power of Monte Carlo Method: Increasing the Number of Iteration

Abstract

Iteration number in Monte Carlo simulation method used commonly in educational research has an effect on Item Response Theory test and item parameters. The related studies show that the number of iteration is at the discretion of the researcher. Similarly, there is no specific number suggested for the number of iteration in the related literature. The present study investigates the changes in test and item parameters resulting from the changes in MC simulation studies based on Item Response Theory. In this respect, the required number of iterations is determined through IRT three-parameter logistics model test and item parameters under different conditions regarding sample size, item number, and parameter restrictions. The results indicate that estimate error can be lowered to a specific point and the test information increases as the number of iterations is increased and that the required number of iterations decreases as the sample size gets larger. However, it is also observed that the required number of iterations increases when intervals that would restrict parameters during data generation process are defined. It is concluded that the number of iterations has a significant impact on estimate results in MC studies and that the required number of iterations depends on the number of conditions and their levels. The more complex and featured the conditions are, the higher number of iterations will be required to achieve estimates without errors.

Computational procedures for quality assessment of latent concepts

“…For each simulation experimental design, 200 samples have been selected (Koçak, 2020), and the three multidimensional indices (, and ) have been computed. Here, we establish the notation adopted in describing the simulation algorithms used to perform the sensitivity analysis.…”