Does an incremental approach to implementing programmatic assessment work? Reflections on the change process

Abstract

In 2017, the School of Medicine (Fremantle) of the University of Notre Dame Australia began moving towards programmatic assessment. Programmatic assessment seeks to achieve robust assessment validity through the assessment of a large number of low-stakes activities or data points. These data points exemplify assessment as learning by valuing feedback, discussion and reflection, ultimately leading to deeper student engagement without compromising credible decision-making on student progress. The School adopted an incremental approach to implementing programmatic assessment that included first establishing data-informed mentoring, and then activating a continuous assessment program that contributed simultaneously to student learning and School decision-making. Action research helped understand the impact of the initiative. Re-engineering continuous assessment as an incremental step towards programmatic assessment proved to be problematic. Some ideas are proposed to draw the strands of programmatic assessment together that may be useful for others to chart a more fruitful path.

Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles

“…However, there could also be principles that are easier to implement or address a certain need in a particular organisation or context, which might create effective hybrids. There are examples of such hybrid or incremental approaches (Bala et al 2020;Bate et al 2020) and McDowell reported on the implementation of programme focused assessment in a number of Higher Education institutes in the United Kingdom, which featured horizontal integrative assessment across stages/years of a programme (McDowell 2012). More research on the effects of various implementations of programmatic assessment on outcomes such as learning behaviour and decision making is definitely needed.…”

An evaluation of programmatic assessment across health professions education using contribution analysis

“…Yet early adopters have encountered challenges such as stakeholder resistance (Baartman et al, 2023;Jamieson et al, 2022;Schut et al, 2021) and design choice dilemmas when applying theoretical principles to authentic practice (Baartman et al, 2022;Schut et al, 2021). The continued perception, or use, of summative assessments has proven di cult to shake and undermines assessment's intended learning function (Baartman et al, 2023;Bate et al, 2020;Schut et al, 2021). There is propensity for overassessment which can burden all stakeholders (Bate et al, 2020;Schut et al, 2021).…”

“…The continued perception, or use, of summative assessments has proven di cult to shake and undermines assessment's intended learning function (Baartman et al, 2023;Bate et al, 2020;Schut et al, 2021). There is propensity for overassessment which can burden all stakeholders (Bate et al, 2020;Schut et al, 2021). These implementation challenges may impede widespread adoption and hinder attainment of desired outcomes (Ryan et al, 2023).…”

Assessment approaches in undergraduate health professions education: towards the development of feasible assessment approaches for low-resource settings

“…The concept of ‘data point’ applied in the PA approach refers to information on student performance that is collected from a single assessment method [ 8 , 12 ]. Single data points are used in formative assessment to provide constructive feedback, which guides learning [ 13 , 16 ]. The progressive accumulation of multiple data points becomes the premise of pass-or-fail decisions reached by a group of assessment experts [ 3 ].…”