Effectiveness of Collaboration in VET: Measuring Skills for Solving Complex Vocational Problems With a Multidimensional Authentic Technology-Based Assessment

Abstract

Context:Collaboration and digitization in work and production process have been of scientific interest in the research field of vocational education and training for decades. So far, there is a lack of empirical evidence on the extent to which the performance of collaborative problem-solving can outperform individual problem-solving. An appropriate instrument for measuring collaborative performance appropriately has been lacking to this point: It is challenging to (1) collect data on interaction in collaboration in a standardized way and (2) to assign the individual performances to the group performance. In a web-based office simulation, this study examined the two-dimensional construct of collaborative problem-solving ability, which consists of cognitive and social components, and explored the effectiveness of collaboration.
 Methods:The test settings are systematically varied between individual and collaborative problem-solving in a domain-typical problem scenario in the economic domain. A theory-driven two-dimensional authentic instrument with a complex commercial problem in a web-based office simulation was applied to measure the performance in these two test settings. Multi-Group Confirmatory Factor Analysis was conducted to test for measurement invariance across test settings, the two-dimensional RASCH-Model incorporating between-item multidimensionality with correlated dimensions was used for ability estimation. ANOVA test was conducted to determine if there is a statistically significant difference with respect to the ability between the individual and collaborative test settings.
 Findings:The two-dimensional construct of collaborative problem-solving ability with correlated dimensions showed better fit than the unidimensional construct. The scalar measurement invariance was established after excluding three items. ANOVA tests confirmed that the collaborative setting enhances the problem-solving ability of learners significantly in both the cognitive component and social component, and the ability in cognitive component was more fostered.
 Conclusion:The findings suggested the validation of this multidimensional authentic instrument to measure the performance in a collaborative work environment in economic domains. The results showed the effectiveness of collaboration with a computer-based agent. In the practice of vocational training and education, teachers can use this instrument for the evaluation of the effectiveness of collaboration.

Fostering Innovative Learning and Satisfaction in Virtual Teamwork: Shedding Light on Apprentices

“…Practical Implications VET is supposed to prepare young adults for the requirements of their current and future workplaces. However, the digital transformation has changed the way we work and how humans interact in an unprecedented way (European Commission, 2013;Paeßens et al, 2023). Particularly the area of collaboration seems to be impacted rigorously (Schwendimann et al, 2018).…”

“…Major changes become apparent particularly in the world of work. The rapid transformation of the economy and work processes severely affect how work is performed in organizations (Dobricki et al, 2020;Paeßens et al, 2023). One important change regards the ways of working together: Collaboration in (global) teams in a technology-enhanced working environment is becoming increasingly important (Noble & Billet, 2017;Schlicht, 2019).…”

Research on Social Skills in Indonesia: A Systematic Literature Review in Reputable Journals

“…Hassan et al (2023), Sneed et al (2023) and Tong et al (2023) social skills are defined as a series of important abilities that students must have, becoming one of the main foundations for their success. In this context, social skills involve the ability to collaborate with classmates, communicate effectively, be actively involved in learning, show high empathy, build positive social relationships, and handle problems (Paeßens et al, 2023;Sharma et al, 2022;Witti et al, 2023). Through this combination, students can form a solid foundation of social skills for their future.…”