Performance assessment and standardization in higher education: A problematic conjunction?

Abstract

Background There is unrealized potential in higher education for greater use of performance assessment, particularly in support of teaching and learning: Well‐designed performance tasks can elicit evidence regarding what students know and can do with respect to complex learning objectives. At the same time, there is some pressure, at least in the United States, to widen usage of standardized assessments in order to facilitate comparisons among institutions. Aims and methods We distinguish assessments requiring extended performances of some kind from those involving other response formats. Taking student‐produced essays as a paradigmatic example, we examine the processes of assessment design and scoring as they pertain to the validity of the inferences drawn from student work products. We also briefly review the history of standardized assessments and the roles they have played in education. Further, we describe how advances in assessment theory and practice have permitted substantial relaxation in the implementation of standardization. Results Our analysis of these issues in the higher education context draws on empirical evidence regarding both faculty usage of performance tasks (particularly as a means for enhancing the construct validity of an overall assessment) and faculty resistance to the imposition of externally developed, standardized assessments. We also describe efforts undertaken in the United States to strengthen faculty expertise in assessment design, scoring, and interpretation. Conclusions We conclude that performance tasks have an important role to play in higher education, especially in view of the complexity and depth of the learning goals set for the students. For many purposes, they cannot be replaced by simpler, cheaper forms of assessment. Efforts to facilitate comparability in evaluating institutional outcomes, however, have not borne much fruit despite the substantial investments made. We argue that resources would be better directed at improving the quality of assessments developed and employed by college faculty and suggest how that might be accomplished.

Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation

“…They all suffer from moderate to severe construct underrepresentation; that is, they fail to capture important facets of the CT construct such as perspective taking and communication. High fidelity performance tasks are viewed as more authentic in that they provide a problem context and require responses that are more similar to what individuals confront in the real world than what is offered by traditional multiple-choice items (Messick, 1994;Braun, 2019). This greater verisimilitude promises higher levels of construct representation and lower levels of construct-irrelevant variance.…”

“…Such performance tasks have the capacity to measure facets of CT that are imperfectly assessed, if at all, using traditional assessments (Lane and Stone, 2006;Braun, 2019;Shavelson et al, 2019). However, these assertions must be empirically validated, and the measures should be subjected to psychometric analyses.…”

“…Of particular importance is the evaluation of the response with respect to the last dimension of the scoring rubric; namely, the overall coherence and persuasiveness of the argument, building on the explicit or implicit characteristics related to the first five dimensions. The scoring process must be monitored carefully to ensure that (trained) raters are judging each response based on the same types of features and evaluation criteria (Braun, 2019) as indicated by interrater agreement coefficients.…”

“…In this context, one advantage of the iPAL framework is that it can provide valuable guidance and an explicit structure for faculty in developing performance tasks for both instruction and formative assessment. When faculty design assessments, their focus is typically on content coverage rather than other potentially important characteristics, such as the degree of construct representation and the adequacy of their scoring procedures (Braun, 2019).…”

The Role of Students’ Beliefs When Critically Reasoning From Multiple Contradictory Sources of Information in Performance Assessments

“…Their suitability for assessing CR based on multiple sources of information in particular, may be limited by challenges in objective scoring and the brevity of the prompt (Zlatkin-Troitschanskaia et al, 2019b). While ecological validity in particular is especially limited in standardized tests (Braun, 2019), CR can be more adequately measured through performance assessments that simulate the complex environment students find themselves in 'in everyday life, ' and provide an addition to standardized measures, as they are better suited to reflect current contexts and learning conditions inside and outside of higher education (Oliveri and Mislevy, 2019;Shavelson et al, 2019).…”