Programmatic assessment design choices in nine programs in higher education

Abstract

In programmatic assessment (PA), an arrangement of different assessment methods is deliberately designed across the entire curriculum, combined and planned to support both robust decision-making and student learning. In health sciences education, evidence about the merits and pitfalls of PA is emerging. Although there is consensus about the theoretical principles of PA, programs make diverse design choices based on these principles to implement PA in practice, fitting their own contexts. We therefore need a better understanding of how the PA principles are implemented across contexts—within and beyond health sciences education. In this study, interviews were conducted with teachers/curriculum designers representing nine different programs in diverse professional domains. Research questions focused on: (1) design choices made, (2) whether these design choices adhere to PA principles, (3) student and teacher experiences in practice, and (4) context-specific differences between the programs. A wide range of design choices were reported, largely adhering to PA principles but differing across cases due to contextual alignment. Design choices reported by almost all programs include a backbone of learning outcomes, data-points connected to this backbone in a longitudinal design allowing uptake of feedback, intermediate reflective meetings, and decision-making based on a multitude of data-points made by a committee and involving multi-stage procedures. Contextual design choices were made aligning the design to the professional domain and practical feasibility. Further research is needed in particular with regard to intermediate-stakes decisions.

An evaluation of programmatic assessment across health professions education using contribution analysis

“…Data points for programmatic assessment may need to shift towards a greater reliance on collective competency assessment with evidence of health impact. Our ndings build upon the work of others(Baartman et al, 2022;Torre et al, 2021) by sequencing the critical steps, and revealing the in uential factors and threats. This knowledge can be used to prioritise resources for maximal outcomes and support creative solutions to design dilemmas, guided by educator priorities and values(Cleland et al, 2020).…”

“…Studies a rm the capacity for early detection and supportive remediation of underperformance (Schut et al, 2021). Yet early adopters have encountered challenges such as stakeholder resistance (Baartman et al, 2023;Jamieson et al, 2022;Schut et al, 2021) and design choice dilemmas when applying theoretical principles to authentic practice (Baartman et al, 2022;Schut et al, 2021). The continued perception, or use, of summative assessments has proven di cult to shake and undermines assessment's intended learning function (Baartman et al, 2023;Bate et al, 2020;Schut et al, 2021).…”

“…External factors become entwined, exerting unrecognised in uence on the system. The nexus between these in uences -principles, cultures, people, external factors -coalesce unpredictably to make programmatic assessment non-standardised, context-speci c, and inherently complex (Baartman et al, 2022;Torre et al, 2020). This complexity makes evaluation challenging and yet, as popularity grows, there is a need to understand the mechanisms upon which successful implementation is contingent (Torre et al, 2020;Torre et al, 2021;Van Melle et al, 2021).…”

“…Emerging from theory (van der Vleuten & Schuwirth, 2005), the literature now widely describes the principles underpinning programmatic assessment (Heeneman et al, 2021). The recent proliferation of implementation within medicine, veterinary science, dietetics, paramedicine, dentistry, teaching and communication science (Baartman et al, 2022;Torre et al, 2021) demonstrates diffusion of theory into practice. Programmatic assessment is fast becoming the prevailing approach to competency-based assessment (Caretta-Weyer et al, 2024;…”

The infrastructural role of education in the social and investment model of economic growth: the specifics of emerging markets

“…According to this concept, the infrastructural role of education in the social and investment model of economic growth is substantiated and emphasized in the works Aleixo et al (2020), Finnveden et al (2020), Neelam et al (2020), andPedro et al (2020). The specifics of higher education's development in emerging markets are studied in the works Adeyinka-Ojo et al (2020), Baartman et al (2022), Fuchs (2022, Gonzales-Valdivia et al (2022), Hassan (2020), Lohberger and Braun (2022), Saqib et al (2020).…”