Crafting Engaging Science Environments is a high school chemistry and physics project-based learning intervention that meets Next Generation Science Standards performance expectations. It was administered to a diverse group of over 4,000 students in a randomized control trial in California and Michigan. Results show that treatment students, on average, performed 0.20 standard deviations higher than control students on an independently developed summative science assessment. Mediation analyses show an indirect path between teacher- and student-reported participation in modeling practices and science achievement. Exploratory analyses indicate positive treatment effects for enhancing college ambitions. Overall, results show that improving secondary school science learning is achievable with a coherent system comprising teacher and student learning experiences, professional learning, and formative unit assessments that support students in “doing” science.

In response to the call for promoting three-dimensional science learning (NRC, 2012), researchers argue for developing assessment items that go beyond rote memorization tasks to ones that require deeper understanding and the use of reasoning that can improve science literacy. Such assessment items are usually performance-based constructed responses and need technology involvement to ease the burden of scoring placed on teachers. This study responds to this call by examining the use and accuracy of a machine learning text analysis protocol as an alternative to human scoring of constructed response items. The items we employed represent multiple dimensions of science learning as articulated in the 2012 NRC report. Using a sample of over 26,000 constructed responses taken by 6700 students in chemistry and physics, we trained human raters and compiled a robust training set to develop machine algorithmic models and cross-validate the machine scores. Results show that human raters yielded good (Cohen's k = .40-.75) to excellent (Cohen's k > .75) interrater reliability on the assessment items with varied numbers of dimensions. A comparison reveals that the machine scoring algorithms achieved comparable scoring accuracy to human raters on these same items. Results also show that responses with formal vocabulary (e.g., velocity) were likely to yield lower machine-human agreements, which may be associated with the fact that fewer students employed formal phrases compared with the informal alternatives.

When the COVID‐19 pandemic struck, research teams in the United States and Finland were collaborating on a study to improve adolescent academic engagement in chemistry and physics and the impact remote teaching on academic, social, and emotional learning. The ongoing “Crafting Engaging Science Environments” (CESE) intervention afforded a rare data collection opportunity. In the United States, students were surveyed at the beginning of the school year and again in May, providing information for the same 751 students from before and during the pandemic. In Finland, 203 students were surveyed during remote learning. Findings from both countries during this period of remote learning revealed that students' academic engagement was positively correlated with participation in hands‐on, project‐based lessons. In Finland, results showed that situational engagement occurred in only 4.7% of sampled cases. In the United States, students show that academic engagement, primarily the aspect of challenge, was enhanced during remote learning. Engagement was in turn correlated with positive socioemotional constructs related to science learning. The study's findings emphasise the importance of finding ways to ensure equitable opportunities for students to participate in project‐based activities when learning remotely.

No abstract