Designing a framework and validating a tool for evaluating the educational quality of serious games: a meta-synthesis

Abstract

In recent decades, at the same time as the quantitative growth in the industry of serious games, its quality has also been the focus of investors, researchers, and developers. Therefore, the purpose of this research is to design a framework and validate a tool for evaluating the educational quality of serious games. (i.e., a questionnaire). Evaluation frameworks and questionnaires are fundamental tools for designing and developing serious games. The method of this research was meta-synthesis with a mixed approach. Based on this, 5807 articles were identified during the years 1995 to 2021, and finally, 29 articles were selected for analysis. By analyzing these articles, basic dimensions, components, and indicators were extracted and turned into an evaluation tool using thematic analysis with the purpose of validation. First, the content validity of this tool was obtained by purposive sampling technique with the participation of 30 serious game experts. Then, for the validity of the constructs, a sample equal to the community was assumed, and the tool was sent to the members of the National Computer Games Foundation, and 537 people participated in completing the tool. Finally, the data were analyzed using confirmatory factor analysis. The findings of first and second-order factor analysis confirmed all dimensions, components, and indicators of the tool with a factor load (above 0.40) and a significant coefficient (above 1.96). Therefore, this meta-synthesis led to the design of the framework and the validation of the educational quality evaluation tool of serious games with 4 dimensions, 25 components, and 138 indicators. We argue that the proposed framework and tool are able to evaluate the educational quality of a serious game and cover its design project end to end. We thus propose them as a suitable resource for researchers and developers of serious games.

Role-Playing in Teacher Education with InCoLearn and its Qualitative Usability

“…Usability research is important for the fundamental evaluation of the quality aspects of any software product. It is furthermore an important endeavor in DGBL research [25] and all major stages of game production, such as pre-production, production [26] as well as post-production and games user research (GUR) in general [27]- [28] to enhance the gaming [29] and learning experience. Furthermore, it ensures the utility and acceptability of serious education games from pedagogical, technical, and contextual standpoints [30].…”