Comparing objective and subjective learning curves: Judgments of learning exhibit increased underconfidence with practice.

Abstract

When participants studied a list of paired associates for several study-test cycles, their judgments of learning (JOLs) exhibited relatively good calibration on the 1st cycle, with a slight overconfidence. However, a shift toward marked underconfidence occurred from the 2nd cycle on. This underconfidence-with-practice (UWP) effect was very robust across several experimental manipulations, such as feedback or no feedback regarding the correctness of the answer, self-paced versus fixed-rate presentation, different incentives for correct performance, magnitude and direction of associative relationships, and conditions producing different degrees of knowing. It was also observed both in item-by-item JOLs and in aggregate JOLs. The UWP effect also occurred for list learning and for the memory of action events. Several theoretical explanations for this counterintuitive effect are discussed.

The intricate relationships between monitoring and control in metacognition: Lessons for the cause-and-effect relation between subjective experience and behavior.

“…A Condition ϫ Presentation ANOVA yielded F(1, 26) ϭ 1.39, MSE ϭ 0.11, p ϭ .25, for condition; F(3, 78) ϭ 8.04, MSE ϭ 0.05, p Ͻ .0001, p 2 ϭ .24, for presentation; and F Ͻ 1 for the interaction. Thus, practice improved monitoring resolution, consistent with previous results (King et al, 1980;Koriat, 1997;Koriat et al, 2002;Mazzoni et al, 1990). The similarity of the results for the self-and other-paced conditions reinforces the conclusion that the effective mnemonic cue in the case of self-paced participants is not study time per se but rather memorizing effort, which can also be used by the otherpaced participants.…”

“…Such regulation should have an adaptive value in terms of improving the effectiveness of cognitive performance. According to Story 1, then, metacognitive feelings play a mediating role similar to that of fear in the first option mentioned earlier: Once such feelings have been formed on the basis of whatever cues available, they can be 1 This aspect of metacognitive accuracy, which is labeled resolution or relative accuracy (see Koriat et al, 2002;Nelson & Dunlosky, 1991), is commonly indexed by a within-participant gamma correlation between metacognitive judgments and actual memory performance (Nelson, 1984). In contrast to resolution, calibration (or absolute accuracy), which refers to the correspondence between mean metacognitive judgments and mean actual performance and reflects the extent to which metacognitive judgments are realistic, tends to be quite poor, generally exhibiting overconfidence (see Metcalfe, 1998).…”

Confidence Calibration in a Multi-year Geopolitical Forecasting Competition

“…On the one hand, there are circumstances in which new information increases confidence more than accuracy, exacerbating overconfidence (Hall et al 2007, Heath and Gonzalez 1995, Oskamp 1965. On the other hand, there are other circumstances in which increasing expertise can increase accuracy faster than confidence, reducing overconfidence (Koriat et al 2002). The parallel development of confidence and accuracy over time implies that our forecasters' judgments navigated adeptly between the Scylla and Charybdis of these twin risks.…”

The Dynamics of Learning and Allocation of Study Time to a Region of Proximal Learning.

“…A trenchant observation by Kahneman (1973; also see Koriat, Sheffer, & Ma'ayan, 2002) also suggests that, in contrast to the return-per-second data, people's strategies of selectively studying the items of medium difficulty and not devoting too much time to the easy items may not have been misguided. Kahneman argued that certain materials only allow so much attention.…”