Do Role-Playing Simulations Generate Measurable and Meaningful Outcomes? A Simulation’s Effect on Exam Scores and Teaching Evaluations

Abstract

Role-playing simulations are frequently claimed to be effective pedagogical tools in the teaching of international relations (IR); however, there is a surprising lack of empirical evidence on their classroom utility. The assessment of simulations remains mostly anecdotal, and some recent research has found little to no statistically significant improvements in quantitative measures of academic performance among students who participated in them [for example, International Studies Perspectives (2006), vol. 7, pp. 395; International Studies Perspectives (2008), vol. 9, pp. 75-89]. Scant research has been conducted on how role-playing simulations might affect students' perceptions of the instructor's teaching. This paper investigates whether a simulation had statistically significant effect on students' exam scores in an IR course or on student teaching evaluation scores.Role-playing simulations are often described as positive learning experiences for students. By their very nature, simulations allow students to participate more actively in the learning process and gain a more thorough understanding of course material. A better grasp of course content would seem to result in better academic performance and more favorable impressions of the instructor's teaching by students. In the Fall 2006, Spring 2007, and Fall 2007 semesters, I taught seven sections of the same introductory international relations (IR) course at a private university with an undergraduate enrollment of less than 5,000 students. Four of these sections participated in a role-playing simulation on the Middle East. During the same three semesters, three other sections of the course did not participate in the simulation. In contrast to what is commonly expected from the use of simulations, I noticed that there seemed to be no difference in academic performance between students who participated in the simulation and those who did not. The simulation also appeared to be associated with lower teaching evaluation results. I decided to investigate what statistically significant relationships, if any, might have existed between students' involvement in the simulation, their performance on exams, and their evaluation of my teaching. 1 1 Data was analyzed after students' final grades had been submitted. Identifying information was removed from the data set to protect student anonymity. Ó 2010 International Studies AssociationInternational Studies Perspectives (2010) 11, 51-60.

Missing the Trees for the Forest?: Learning Environments Versus Learning Techniques in Simulations

“…Stroessner et al (2009:614) found that in comparison to traditional instruction, Reacting to the Past role play exercises produced no statistically significant benefit in writing skills among first-year college students, and that the benefit to students' rhetorical skills was "marginal." Raymond (2010) found no statistically significant improvements in exam scores among students who participated in a role-playing simulation compared to students who received traditional lectures and assignments in an international relations course.…”

Is It Better to Be Feared Than Loved? Investigating the Efficacy of Different Teaching Methods on the Learning of Specific Content

“…Recent studies using this technique to enhance student learning in introductory-level political science courses attempted to measure its eff ect on student learning in diff erent ways. Raymond (2010), for example, examined the eff ect of a simulation exercise on exam scores; Williamson and Gregory (2010) sought evidence that multiple PBL exercises lead to better critical thinking, use of sources, and understanding of political institutions in student papers. In a simpler approach, Centellas and Love (2012) looked for improved performance on a battery of multiple-choice questions testing specifi c knowledge that should have been reinforced in a fourweek PBL exercise.…”

Outsourcing Learning: Is the Statecraft Simulation an Effective Pedagogical Alternative?

“…As Krain and Lantis (2006:399, 400) have shown, "very few studies confirm our experiences (and convictions) that however, there were no statistically significant improvements in exam scores among simulation participants compared to students who received traditional lectures and assignments. Although students indicated that they thought the simulation was a useful educational experience, participation in the simulation was associated with lower scores on students' evaluations of my teaching (Raymond 2010).…”