Development of the Water Instrument: a comprehensive measure of students’ knowledge of fundamental concepts in general chemistry

Section: Methodsmentioning

confidence: 99%

Students’ interactive engagement, academic achievement and self concept in chemistry: an evaluation of cooperative learning pedagogy

Udu¹,

Nmadu²

2023

“…A convenience sampling approach was used for this study within two large, public, research-intensive universities. The sample pool consisted of students who had completed the twosemester sequence of General Chemistry and completed the Water Instrument assessment for General Chemistry (Balabanoff et al, 2022). Interview participants were recruited at the end of the assessment and were compensated for their time with a gift card.…”

Section: Participantsmentioning

confidence: 99%

“…Preliminary interviews were collected in Fall 2019 as part of an assessment development study investigating validity and reliability of a general chemistry assessment which utilized a two-tier question format (Balabanoff et al, 2022). Response process validity (RPV) interviews were carried out to collect novice content validity evidence for assessment items.…”

Section: Data Collectionmentioning

confidence: 99%

“…While this is a common assessment format that students will likely see in future courses, assumptions outside of this context cannot be made. Additionally, the multiplechoice questions presented in interviews were items from the Water Instrument assessment (Balabanoff et al, 2022) that students had previously interacted with. Interviews took place shortly after the course had ended.…”

Section: Limitationsmentioning

confidence: 99%

See 1 more Smart Citation

“I guess it was more than just my general knowledge of chemistry”: exploring students’ confidence judgments in two-tiered assessments

Koevoets-Beach,

Julian,

Balabanoff

2023

Self Cite

Two-tiered assessment structures with paired content and confidence items are frequently used within chemistry assessments to stimulate and measure students’ metacognition. The confidence judgment is designed to promote students’ reflection on their application of content knowledge and can be characterized as calibrated or miscalibrated based on their accuracy. Previous studies often attributed students’ miscalibrated confidence rankings to metaignorance, however, in this qualitative study, interviews with general chemistry students were thematically analysed to provide a more robust understanding of the processes and factors students use when engaging with these metacognitive prompts in a chemistry assessment. Both calibrated and miscalibrated confidence judgments were observed independent of accuracy. Students who provided miscalibrated confidence judgments often used unreliable metrics such as processing fluency which can mimic content mastery whereas students who provided more accurate evaluations of their confidence relied more heavily on their stable understanding of chemistry concepts. Many students cited previous experiences, underlying self-efficacy beliefs, and/or the use of test-taking strategies which negatively or positively impacted their confidence. These findings suggest that the confidence tier is indeed capturing students’ self-assessment, however, students’ confidence judgments are based on a range of factors independent of content knowledge which may impede on the utility of this metacognitive tool for students, researchers, and instructors.

“…Rasch analysis is a statistical procedure and mathematical modeling technique within item response theory that is centered on the existence of a latent trait (Boone, 2016). Rasch analysis has been used extensively in the development and assessment of instruments (Wren and Barbera, 2014;Taskin et al, 2015;He et al, 2016;Lu and Bi, 2016;Nedungadi et al, 2019;Lu et al, 2020;Balabanoff et al, 2021). Rasch analysis is particularly beneficial in the analysis of instruments because it allows for generalization of the results to the greater population, whereas analysis using CTT is sample dependent and thus not generalizable between samples.…”

Section: Rasch Analysismentioning

confidence: 99%

Development of the Assessment of Student Knowledge of Green Chemistry Principles (ASK-GCP)

Grieger

Schiro

Leontyev

2022