An integrated STEM-based mathematical problem-solving test: Developing and reporting psychometric evidence

Abstract

Science, technology, engineering, and mathematics (STEM) problem-solving is necessary to be infused into the classroom. Nevertheless, the criticism of underrepresented mathematics in STEM problem-solving assessment is an issue. In this study, we develop and investigate the psychometric evidence of an integrated STEM-based mathematical problem-solving test. The product of the test was a mathematical essay test that contains three scientific scenarios related to the environment in every middle school grade. The mathematical contents were integrated into engineering-based design using the technology. Three experts filled an assessment sheet to assess content validity, which was analyzed using a content validity index (CVI) and intraclass correlation coefficient (ICC). The result of content validity revealed that overall items were valid and reliable. The construct validity was examined using the Rasch analysis from the data of Grades 7–9 students in Indonesia (n = 286). The construct of all scenarios and prompting items indicated fit with various difficulty levels and acceptable discrimination value. Nevertheless, four prompting items were reported as misfit based on unweighted mean square value. The recommendation for improvement is emphasized in the language clarity aspect. The inter-rater reliability was also declared good. A further study is suggested to provide a computer-based test.

Design and content validity of mathematics creative problem-solving ability instrument for junior high school students

“…The content validation process in this research was carried out by seven expert validators experienced in mathematics education and measurement. Content validation helps evaluate expert assessments of the instruments being developed (Aktas & Tabak, 2018;Amalina & Vidákovich, 2022;Kahveci et al, 2016). The findings regarding content validation align with research results that show that quantitative analysis of data through expert assessment helps reveal the strengths and weaknesses of the instrument.…”

Stem in Indonesian Mathematics Education

“…The results of the analysis of 23 articles about STEM, 9 articles that developed student activity sheets based on measured abilities, The correct use of Activity Sheets in the learning process can improve learning outcomes, students' thinking abilities such as creativity, critical thinking, reasoning, understanding concepts, and student communication (Saputra et al, 2020;Argianti, et al, 2021;Arivina, 2020;Novaliyosi, et al, 2021;Sukendra, et al, 2020;Komarudin, et al, 2021;Sulistiawati, et al, 2021;Amalina & Vidakovich, 2022;Rohimah et al, 2022). A resume of STEM articles published in 2020-2024 based on the media developed can be seen in Figure 4.…”

“…The results of the analysis of 23 articles about STEM can be seen in Figure 5. (Amalina & Vidakovich, 2022;Saputra et al, 2020;Arifin et al, 2020;Arivina & Jailani, 2020;Novaliyosi et al, 2021;Krisdiana et al, 2020;Sukendra & Widana, 2023).…”

Exploring Pre-service Teachers’ Cognitive Process in Solving Probability Problems Related to the Maritime Context of the Riau Archipelago

“…Various studies have been conducted to seek solutions for enhancing learners' thinking abilities. These studies investigate factors influencing problem-solving abilities [1][2][3][4], effective learning models to develop problem-solving skills [5][6][7], effective learning to promote problem-solving skills [8] and more.…”

“…Problem-solving is a complex cognitive process that demands learners to understand the problems, and select and apply relevant scientific concepts to solve them [4]. Learners need systematic and logical thinking to comprehend, analyze, and interpret information in the problems.…”