Effects of lecture fluency and instructor experience on students’ judgments of learning, test scores, and evaluations of instructors.

Abstract

Students' judgments of learning (JOLs) are often driven by cues that are not diagnostic of actual learning. One powerful cue that can mislead JOLs is lecture fluency-the degree to which an instructor delivers a smooth, confident, and well-polished lecture. Lecture fluency often inflates JOLs, but has no effect on actual learning. The limited research so far, however, has not systematically explored the role of instructor experience, which may moderate the effects of lecture fluency. In two experiments, students viewed a video-recorded lecture of a fluent or disfluent lecture, and beforehand were informed that the instructor was experienced or inexperienced. Afterward, students made a JOL estimating how much they had learned, answered several evaluation questions, and took a test. Significant effects of lecture fluency, but not instructor experience, occurred whereby lecture fluency inflated JOLs but not test scores. As well, students more often based their JOLs on lecture fluency than instructor experience. The fluent lecture received more favorable evaluations than the disfluent lecture, including students' increased interest in the material and willingness to attend class, suggesting that fluent instruction might benefit learning in indirect ways that are not reflected in test scores. Public Significance StatementStudents' evaluations of their own learning can be misled by cues that are highly intuitive but unreliable. The fluency with which an instructor delivers a lecture (i.e., in a smooth and well-polished manner), but not the perceived experience of the instructor, coincides positively with students' impressions of the instructor and how much they feel they have learned, but does not reliably enhance learning, raising questions about the influence of certain instructor characteristics that have long been considered qualities of effective teaching.

Gender Bias Interacts With Instructor Disfluency to Negatively Affect Student Evaluations of Teaching

“…Given the extensive research on gender bias, we hypothesized that the female instructor will be rated lower than the male instructor in both the fluent and disfluent conditions. Consistent with the research by Carpenter et al (2013Carpenter et al ( , 2016Carpenter et al ( , 2020a, we further hypothesized that participants in the fluent condition would overestimate their learning and rate the instructor higher compared to the disfluent condition. Fluency could have no effect on quiz performance (consistent with work done in the Carpenter lab); however, because the disfluency of the current study involves more verbal, as opposed to visual, disfluency, we may find that participants perform worse on the quiz in the disfluent condition compared to the fluent condition.…”

“…Free recall tends to be more challenging because the correct answer needs to be retrieved, but only recognition of the correct response is necessary in multiple choice (Kintsch, 1970). Carpenter et al (2020a) also used multiple choice, but their quiz consisted of 30 items as opposed to 10 in the present experiment. Indeed, performance in their study was around 55% whereas our participants neared 70% correct on the final quiz.…”

“…The findings of previous research Carpenter et al, 2020a) leave two questions left unanswered. The first is whether verbal, as opposed to visual, forms of disfluency influence student perceptions of learning and the instructor.…”

“…Specifically, students rated the fluent instructor significantly higher on organization, knowledge, preparedness, and effectiveness compared to the disfluent instructor. Recently, this finding has not been shown to be moderated by the instructor's apparent experience (Carpenter et al, 2020a). Carpenter and colleagues have primarily manipulated visual forms of disfluency (e.g., poor eye-contact and slouching).…”

Is a picture really worth a thousand words? Evaluating contributions of fluency and analytic processing in metacognitive judgements for pictures in foreign language vocabulary learning

“…This “metacognitive illusion” (Rhodes & Castel, 2009) has been shown for visual stimuli that are presented in larger font compared with smaller font (Kornell, Rhodes, Castel, & Tauber, 2011; Rhodes & Castel, 2008), in clearer font compared with blurred font (Yue, Castel, & Bjork, 2013), or in type-print font compared with handwritten cursive (Geller, Still, Dark, & Carpenter, 2018). In other domains, overconfidence is greater for auditory stimuli that are presented in louder compared with softer volume (Rhodes & Castel, 2009), and for lectures that are delivered in a manner that is smooth and well-polished compared with fumbling and awkward (Carpenter, Mickes, Rahman, & Fernandez, 2016; Carpenter, Northern, Tauber, & Toftness, 2019; Carpenter, Wilford, Kornell, & Mullaney, 2013; Toftness et al, 2018).…”