Evaluation of competency methods in engineering education: a systematic review

Abstract

The purpose of this systematic review is to evaluate the state-of-the-art of competency measurement methods with an aim to inform the creation of reliable and valid measures of student mastery of competencies in communication, lifelong learning, innovation/creativity and teamwork in engineering education. We identified 99 studies published in three databases over the last 17 years. For each study, purpose, corresponding methods, criteria used to establish competencies, and validity and reliability properties were evaluated. This analysis identified several measurement methods of which questionnaires and rubrics were the most used. Many measurement methods were found to lack competency definitions and evidence of validity and reliability. These show a clear need for establishing professional standards when measuring mastery of competencies. Therefore, in this paper, we propose guidelines for the design of reliable and valid measurement methods to be used by educators and researchers.

Learning through Assessment and Feedback Practices: A Critical Review of Engineering Education Settings

“…3) Assessment practices can enhance engineering students' oral and written communication skills, which are essential for them in their prospective engineering profession (Cruz, Saunders-Smits, & Groen, 2019;Olds et al, 2005). This can be achieved by assessing class seminars, technical reports, etc., and providing qualitative feedback to students to improve the quality of presentation.…”

“…In the context of engineering education, Cruz et al (2019) defines 'assessment' as a process to judge students' level of mastery in; 1) subject knowledge and technical knowledge in their respective engineering disciplines 2) competencies such as communication skills, creative thinking/innovation, lifelong learning and teamwork skills ('employability skills' to enter into the engineering profession)…”

“…They are expected to possess high-quality oral and written communication skills to convey their knowledge (Choudhury, 2019). According to Cruz et al (2019), engineering students are supposed to attain an ability to work as competent team members in an interdisciplinary collaborative environment. Eventually, students are expected to get hold of engineering practices to evolve as engineering professionals (Lal et al, 2017;Subheesh & Sethy, 2018).…”

“…He is of the view that the industrial community across the globe considers theoretical knowledge and technical competencies of engineering graduates as 'satisfactory'. On the other hand, their ability to make sound technical reports and presentations, ability to communicate well, ability to innovate, ability to effectively involve in teamwork, and ability to update their knowledge (through lifelong learning) are not up to the mark as per engineering industry standards (Cruz et al, 2019;Rompelman, 2000). Thus, in order to prepare students for their potential engineering profession, more importance has to be given in developing their competencies in communication, innovation, teamwork, and lifelong learning in addition to technical competency.…”

“…Assessment and feedback have to be carefully designed in a way to ensure students' attainment of the competencies and skills. However, assessing engineering students' proficiency in such competencies and skills is a complex process (Cruz et al, 2019). Development of efficient assessment and feedback practices in engineering education demands 'an extraordinary effort and time commitment' (Vos, 2000).…”