Reactivity to Measures of Metacognition

Abstract

Metacognition is typically measured by collecting self-reported information from participants while they complete a cognitive task. Recent evidence suggests that eliciting such metacognitive information from participants can impact both their metacognitive processes and their cognitive performance. Although there are contradictory findings regarding the magnitude and even the direction of this effect, recent evidence has converged to provide a clearer picture of the mechanisms that determine reactivity. Here, we provide a review of the evidence that measures of metacognition, namely think-aloud protocols, judgments of learning, and confidence ratings, are reactive. We argue that reactivity has important implications not just for the measurement of metacognition, but for metacognition theorizing because reactivity can provide insights into the cues participants use to monitor their performance. Drawing from this synthesis of evidence, we propose a tentative framework for studying reactivity that integrates cue processing accounts of reactivity with existing models of metacognition. We conclude the review by addressing some of the pertinent questions yet to be comprehensively addressed by reactivity research, including how researchers should best address issues of reactivity when using experimental designs.

Metacognitive Awareness Scale, Domain Specific (MCAS-DS): Assessing Metacognitive Awareness During Raven’s Progressive Matrices

“…To enable the measurement of metacognitive awareness, a domain-specific, Raven's SPM-focussed questionnaire was constructed, and then administered immediately following the completion of the focal task. As an offline measure, the construction of the current scale addresses the often-cited problems with online metacognition measurements which are, the disruption of optimal task performance, and the reactivity exhibited when eliciting metacognitive information (Double and Birney, 2019). In addition, there is the issue of metacognitive questionnaires being too generic and not reflecting the cognition involved in the specific task (Allon et al, 1994).…”

“…SPM, as an often-used reasoning task which is also used for the study of intelligence, has therefore its own unique parameters that are best understood using a task-specific scale. This MCAS-DS minimizes disruption to, and impact of, metacognition measure on task performance, and also minimizes reactivity in participants in its design by being an offline measure (Garner and Alexander, 1989;Zakay, 1996;Double and Birney, 2019).…”

“…Further validation of the scale using a larger sample, and using it with other metacognitive scales is recommended. A comparison between how well this scale performs against other scales will be helpful even though the current scale already addresses important theoretical (Thompson, 2000) and measurements issues (Garner and Alexander, 1989;Zakay, 1996;Double and Birney, 2019) not addressed by other known generic metacognitive scales. The sample utilized in the current study encompasses a broad range of individuals of different ages and background but it is less than ideal in terms of sample size.…”

“…Furthermore, there is the assertion that the type of metacognitive measure employed may moderate the metacognition-intelligence relationship (Ohtani and Hisasaka, 2018). Even the elicitation of participants' responses to metacognition measures will alter the processes for which the measures were constructed (Double and Birney, 2019). Together, these make the accurate and valid measure of metacognition, an important issue for this field.…”

Sophisticated Statistics Cannot Compensate for Method Effects If Quantifiable Structure Is Compromised

“…One of the core method effects of concern for psychology is the potential for test-takers to react in some way to the fact that they are being measured (Double and Birney, 2019b). This effect has been referred to as "reactivity."…”