Development and Evaluation of a Survey to Measure Student Engagement at the Activity Level in General Chemistry

Abstract

Student engagement is an important consideration when incorporating active learning activities into a classroom. To facilitate the large-scale assessment of students' engagement in activities, a survey measure must first be developed and evaluated. Therefore, the goal of this study was to create a self-report measure of student engagement for use with active learning activities in general chemistry classes. The Activity Engagement Survey (AcES) was modified from an existing survey of engagement of middle and high school science students that contained behavioral, cognitive, emotional, and social engagement items. Multiple rounds of response process interviews and factor analyses were used to modify the measure and provide evidence of validity for the data collected. The results provided support for the presence of a bifactor model, where an overall general engagement factor and the individual dimensions of engagement were simultaneously modeled. Additionally, support was found for combining behavioral and cognitive components into a single factor due to a large overlap in students' perceptions between the constructs.

Exploring Student Perceptions of Behavioral, Cognitive, and Emotional Engagement at the Activity Level in General Chemistry

“…One of the major findings of this study was the conflation in how students perceive behavioral and cognitive engagement. This finding, combined with support from our quantitative study, 29 implies that these two dimensions of engagement cannot be measured and evaluated separately. Therefore, it is suggested that future studies combine these dimensions when seeking to evaluate students' engagement in learning activities.…”

“…Additionally, a recent quantitative study by Naibert and Barbera in higher-education, which assessed student engagement in active learning activities of general chemistry students, found further evidence for combining behavioral and cognitive engagement into a combined behavioral/cognitive dimension. 29 The results of our qualitative and quantitative studies provide support for students' perceiving a large overlap between behavioral and cognitive engagement when asked specifically to think about the specific activities. Therefore, it may be that students conflate the two dimensions when the focus is on engagement in a specific activity instead of the class as a whole.…”

Investigating Student Engagement in General Chemistry Active Learning Activities using the Activity Engagement Survey (AcES)

“…Engagement was assessed using the BC-E and BC-E-S versions of the Activity Engagement Survey (AcES). 14 The BC-E AcES measure includes 15 items related to students overall engagement, with 10 of those items also related to behavioral/cognitive engagement (e.g., I stayed focused during today's activity) and 5 items related to emotional engagement (e.g., I looked forward to today's activity). The BC-E-S AcES includes 19 items related to students' overall engagement: the same 15 items from the BC-E AcES, as well as 4 items also related to social engagement (e.g., I built on other students' ideas during today's activity).…”

“…Evidence in support of internal structure validity provides confidence that the items relate to the construct as expected. 23,24 The BC-E and BC-E-S AcES bifactor models relate each item to an overall engagement factor, as well as to the relevant behavioral/cognitive, emotional, or social engagement factor (see Naibert and Barbera 14 ). In a previous related study, 14 evidence of internal structure validity was found for the data collected in the remote environment (2020−2021 academic year) (n = 1248) using the BC-E AcES bifactor model with a negative method factor.…”

“…9 In higher-education chemistry courses, social engagement has recently been discovered as a dimension of student engagement in active learning activities in the classroom. 13,14 To begin to explore students' engagement in active learning activities, the two versions of the Activity Engagement Survey (AcES) (see Naibert and Barbera 14 ) were used to evaluate students' engagement in worksheet activities in general chemistry. In the AcES development study, 14 a bifactor model that included the individual dimensions of engagement (i.e., behavioral/cognitive, emotional, and social) as well as an overall engagement dimension, was found to be the most appropriate model.…”