A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use

Bass, Kristin M.; Drits-Esser, Dina; Stark, Louisa A.

doi:10.1187/cbe.15-07-0142

Cited by 20 publications

(25 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We developed the EcoEvo-MAPS questions through an iterative process ( NRC, 2001 ; Adams and Wieman, 2011 ; Bass et al , 2016 ) similar to that used for other biology concept assessments to optimize assessment validity ( Smith et al , 2008 ; Price et al , 2014 ; Couch et al , 2015b ). This approach involved multiple cycles of revision, including feedback from both students and faculty experts ( Table 1 ).…”

Section: Methodsmentioning

confidence: 99%

“…These core concepts were developed following conversations with more than 500 stakeholders in biology education, are supported by several national funding agencies, and overlap with the Next Generation Science Standards ( NGSS Lead States, 2013 ) for K–12 education. While these assessments follow the methodology used for the development of concept inventories ( NRC, 2001 ; Adams and Wieman, 2011 ; Bass et al ., 2016 ), they differ in covering a wide breadth of concepts and are designed to measure student learning in cohorts of students at different time points in the undergraduate program.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates

Summers

Couch

Knight

et al. 2018

LSE

View full text Add to dashboard Cite

A new assessment tool, Ecology and Evolution–Measuring Achievement and Progression in Science or EcoEvo-MAPS, measures student thinking in ecology and evolution during an undergraduate course of study. EcoEvo-MAPS targets foundational concepts in ecology and evolution and uses a novel approach that asks students to evaluate a series of predictions, conclusions, or interpretations as likely or unlikely to be true given a specific scenario. We collected evidence of validity and reliability for EcoEvo-MAPS through an iterative process of faculty review, student interviews, and analyses of assessment data from more than 3000 students at 34 associate’s-, bachelor’s-, master’s-, and doctoral-granting institutions. The 63 likely/unlikely statements range in difficulty and target student understanding of key concepts aligned with the Vision and Change report. This assessment provides departments with a tool to measure student thinking at different time points in the curriculum and provides data that can be used to inform curricular and instructional modifications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates

Summers

Couch

Knight

et al. 2018

LSE

View full text Add to dashboard Cite

show abstract

“…To populate each category in our assessment we selected only the questions from TIPS II with the highest item discrimination index (as reported in Kazeni, 2005), a statistical measure that distinguishes between high performing and low performing examinees for a given assessment. The average item discrimination index of questions selected from TIPS II was 0.4, well above the acceptable range (>0.3; Bass et al, 2016). Student scores on these fifteen TIPS II items were averaged to create the main outcome variable in this study.…”

Section: Research Studymentioning

confidence: 81%

Online, Interactive Tutorials Provide Effective Instruction in Science Process Skills

Kramer

Olson

Walker

2017

Preprint

View full text Add to dashboard Cite

Explicit emphasis on teaching science process skills leads to both gains in the skills themselves and, strikingly, deeper understanding of content. Here, we created and tested a series of online, interactive tutorials with the goal of helping undergraduate students develop science process skills. We designed the tutorials in accordance with evidence-based multimedia design principles and student feedback from usability testing. We then tested the efficacy of the tutorials in an introductory undergraduate biology class. Based on a multivariate ordinary least squares regression model, students that received the tutorials are predicted to score 0.824 points higher on a 15 point science process skill assessment than their peers that received traditional textbook instruction on the same topic. This moderate but significant impact indicates that well designed online tutorials can be more effective than traditional ways of teaching science process skills to undergraduate students. We also found trends that suggest the tutorials are especially effective for non-native English speaking students. However, due to a limited sample size, we were unable to confirm that these trends occurred due to more than just variation in the sampled student group.

show abstract

“…To populate each category in our assessment, we selected only the questions from TIPSII with the highest item discrimination index (as reported in Kazeni, 2005 ), a statistical measure that distinguishes between high-performing and low-performing examinees for a given assessment. The average item discrimination index of questions selected from TIPSII was 0.4, well above the acceptable range (>0.3; Bass et al , 2016 ). Student scores on these 15 TIPSII items were averaged to create the main outcome variable in this study (Supplemental Material 5).…”

Section: Research Studymentioning

confidence: 82%

Design and Assessment of Online, Interactive Tutorials That Teach Science Process Skills

Kramer

Olson

Walker

2018

LSE

View full text Add to dashboard Cite

Explicit emphasis on teaching science process skills leads to both gains in the skills themselves and, strikingly, deeper understanding of content. Here, we created and tested a series of online, interactive tutorials with the goal of helping undergraduate students develop science process skills. We designed the tutorials in accordance with evidence-based multimedia design principles and student feedback from usability testing. We then tested the efficacy of the tutorials in an introductory undergraduate biology class. On the basis of a multivariate ordinary least-squares regression model, students who received the tutorials are predicted to score 0.82 points higher on a 15-point science process skill assessment than their peers who received traditional textbook instruction on the same topic. This moderate but significant impact indicates that well-designed online tutorials can be more effective than traditional ways of teaching science process skills to undergraduate students. We also found trends that suggest the tutorials are especially effective for nonnative English-speaking students. However, due to a limited sample size, we were unable to confirm that these trends occurred due to more than just variation in the student group sampled.

show abstract

A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use

Cited by 20 publications

References 39 publications

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates

Online, Interactive Tutorials Provide Effective Instruction in Science Process Skills

Design and Assessment of Online, Interactive Tutorials That Teach Science Process Skills

Contact Info

Product

Resources

About