One important consideration in designing instruments to measure students' attitudes about a course or an instructor is the influence of the format of the instrument on students' responses. This study contrasted the effects of randomly distributing the items throughout the questionnaire (personality format) versus grouping the items together from the same dimension (achievement format) on students' end-of-term course evaluations. On the last day of class, 376 undergraduate students were randomly assigned to complete anonymously the Academic Motivation Profile (AMP) constructed using either a personality or achievement format. A 2 (Format) x 10 (Class Section) MANOVA of the 12 scales of the course evaluation instrument demonstrated a statistically significant Format effect (Wilks's lambda = .85, p < .001), a statistically significant Section effect (Wilks's lambda =.37, p < .001), and no significant Format x Section interaction (Wilks's lambda = .70, p >.05). Examination of the factor structures using confirmatory factor analysis indicated that the measurement model underlying the AMP fit the data reasonably well for both the achievement and personality formats; however, the achievement format provided a better fit. Based on the statistical results and the fact that the achievement format may be more straightforward, produce less suspicion from the student, and be more time efficient, it should be considered when designing course evaluation instruments.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.