Development and validation of an introductory psychology knowledge inventory.

Abstract

The introductory psychology course is offered by nearly all undergraduate psychology programs. Given its prominence in curricula, there have been calls to develop reliable and valid assessments of introductory psychology knowledge to inform and evaluate instruction. The current research, guided by the American Psychological Association's five content pillars model for the introductory course, developed and evaluated a relatively brief inventory that can be easily administered and scored to assess students' knowledge of introductory psychology. We followed established test development guidelines to write and revise inventory items. Then, we conducted two evaluation studies in which we administered the inventory to introductory psychology courses at two higher education institutions and to a sample of US adults to assess the reliability and validity of the test. Results suggested that the inventory has convergent (high correlations with course exam scores, all ps Ͻ .001), discriminant (low correlations with ACT English and Reading scores, all ps Ͻ .05), and known-groups validity (students that had taken high school psychology scored higher, all ps Ͻ .001). The inventory also demonstrated adequate test-retest reliability across a 1-week time interval, r ϭ .80, p Ͻ .001. In a third study, we surveyed introductory psychology instructors about the inventory. The majority (97.4%) reported that it was representative of typical course content, and 75.7% reported they would use the inventory. We conclude that this introductory psychology knowledge inventory offers a practical, useful, reliable, and valid means of assessing students' knowledge and learning in the introductory course.

Learning Introductory Biology: Students’ Concept-Building Approaches Predict Transfer on Biology Exams

“…For retention items, alpha was 0.49 and 0.40 for the first- and second-semester exams, respectively; for transfer items, alpha was 0.78 and 0.50 for first- and second-semester exams, respectively. However, as argued elsewhere (e.g., Solomon et al ., 2021 ), internal consistency is not wholly appropriate for evaluating the reliability of a test in which the items cover a range of content: One may not expect that a student would perform similarly across that range because the student may have a good understanding of some content but not other content. Test–retest reliability would be a more informative index of the exams’ stability; indeed, see Solomon et al .…”

“…However, as argued elsewhere (e.g., Solomon et al ., 2021 ), internal consistency is not wholly appropriate for evaluating the reliability of a test in which the items cover a range of content: One may not expect that a student would perform similarly across that range because the student may have a good understanding of some content but not other content. Test–retest reliability would be a more informative index of the exams’ stability; indeed, see Solomon et al . (2021 ) for high test–retest reliability of a knowledge assessment (like an exam) in the face of relatively low Cronbach alpha values.…”

A longitudinal multisite study of the efficacy of retrieval and spaced practice in introductory psychology.

“…The lack of variance on this benchmark measure in the original study and in this follow-up suggests the need for a better measure of learning in the Introductory Psychology course (Gurung & Hackathorn, 2018). Whereas a recent measure shows good promise for use in future longitudinal studies of learning (Solomon et al, 2019), attention to capturing learning in a valid and reliable way is clearly needed for scholarship of teaching and learning in psychology.…”

Randomized study strategy intervention in a large introductory psychology course.

“…As a measure of experience with advanced coursework in high school, we calculated an AP proportion score ( M = .23) as the proportion out of five AP science and math subjects (chemistry, biology, physics, calculus, and psychology) that a student had reported to the university and scored at least 4. Additionally, because some students would have had exposure to introductory psychology material in high school, students completed a 30-item multiple-choice psychology knowledge inventory during the first week of the semester (Solomon et al, 2021). There were no significant differences between the two experimental conditions on academic covariates.…”