Comparison of normalized gain and Cohen’s<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:math>for analyzing gains on concept inventories

Nissen, Jayson; Talbot, Robert M.; Thompson, Amreen Nasim; Dusen, Ben Van

doi:10.1103/physrevphyseducres.14.010115

Cited by 72 publications

(45 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These 28 studies do not include the three studies in PER that used MI, which we discussed earlier. We excluded these three articles from the 28 studies that we reviewed because two of them did not report pretest and posttest scores on concept inventories [1,17] and we discuss the third article [2] below.…”

Section: A Missing Data In Per Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Missing data and bias in physics education research: A case for using multiple imputation

Nissen

Donatello

Dusen

2019

Phys. Rev. Phys. Educ. Res.

Self Cite

View full text Add to dashboard Cite

Physics education researchers (PER) commonly use complete-case analysis to address missing data. For complete-case analysis, researchers discard all data from any student who is missing any data. Despite its frequent use, no PER article we reviewed that used complete-case analysis provided evidence that the data met the assumption of missing completely at random (MCAR) necessary to ensure accurate results. Not meeting this assumption raises the possibility that prior studies have reported biased results with inflated gains that may obscure differences across courses. To test this possibility, we compared the accuracy of complete-case analysis and multiple imputation (MI) using simulated data. We simulated the data based on prior studies such that students who earned higher grades participated at higher rates, which made the data missing at random (MAR). PER studies seldom use MI, but MI uses all available data, has less stringent assumptions, and is more accurate and more statistically powerful than complete-case analysis. Results indicated that complete-case analysis introduced more bias than MI and this bias was large enough to obscure differences between student populations or between courses. We recommend that the PER community adopt the use of MI for handling missing data to improve the accuracy in research studies.

show abstract

Section: A Missing Data In Per Studiesmentioning

confidence: 99%

“…The final step pools the estimates using simple combining rules, also known as Rubin's Rules [13], which are described later in Eqs. (1)(2)(3)(4)(5). These pooled results then properly reflect the variation in the original estimates and the variation introduced by the imputation process itself.…”

Section: F Imputation Of Missing Datamentioning

confidence: 99%

Missing data and bias in physics education research: A case for using multiple imputation

Nissen

Donatello

Dusen

2019

Phys. Rev. Phys. Educ. Res.

Self Cite

View full text Add to dashboard Cite

show abstract

“…PER seldom uses multiple imputation [46][47][48] and prefers to use complete-case analysis [37], where researchers discard cases that do not include both a pretest and a post-test. However, research indicates that multiple imputation leads to better analytics than traditional methods such as complete-case analysis [41].…”

Section: Data Collection and Preparationmentioning

confidence: 99%

Modernizing use of regression models in physics education research: A review of hierarchical linear modeling

Dusen

Nissen

2019

Phys. Rev. Phys. Educ. Res.

Self Cite

View full text Add to dashboard Cite

This paper is part of the Focused Collection on Quantitative Methods in PER: A Critical Examination.] Physics education researchers (PER) often analyze student data with single-level regression models (e.g., linear and logistic regression). However, education datasets can have hierarchical structures, such as students nested within courses, that single-level models fail to account for. The improper use of single-level models to analyze hierarchical datasets can lead to biased findings. Hierarchical models (also known as multilevel models) account for this hierarchical nested structure in the data. In this publication, we outline the theoretical differences between how single-level and multilevel models handle hierarchical datasets. We then present analysis of a dataset from 112 introductory physics courses using both multiple linear regression and hierarchical linear modeling to illustrate the potential impact of using an inappropriate analytical method on PER findings and implications. Research can leverage multi-institutional datasets to improve the field's understanding of how to support student success in physics. There is no post hoc fix, however, if researchers use inappropriate single-level models to analyze multilevel datasets. To continue developing reliable and generalizable knowledge, PER should use hierarchical models when analyzing hierarchical datasets. The Supplemental Material includes a sample dataset, R code to model the building and analysis presented in the paper, and an HTML output from the R code.

show abstract

“…Both populations demonstrated similar normalized average learning gains (Hake's g, calculated as (21) . Because Hake's g has been shown to be biased in favor of populations with high pre-assessment scores (e.g., the NCSU population), we also measured effect size (Cohen's d with correction for small sample size, calculated as )) (22,23). Strikingly, the effect size in the UNCP population was 0.82, while the effect size in the NCSU population was 0.50.…”

Section: Evidence Of Student Learningmentioning

confidence: 99%

Building thelacoperon: A guided-inquiry activity using 3D-printed models

Gordy

Sandefur

Lacara

et al. 2020

Preprint

View full text Add to dashboard Cite

AbstractUndergraduate biology courses rely heavily on visual representation of information. Students view images of plants, animals, and microbes, interpret data presented in graphs, and use drawings to understand how cells and molecules interact in three dimensions. Traditional teaching approaches exclude students with visual impairments and disadvantage students with disabilities that affect their interpretation and processing of visual and spatial information as well as students who simply do not identify as “visual learners”. By using new technologies to develop tactile teaching tools (TTTs) that can be employed in classrooms, we aim to create inclusive learning environments and more effectively instruct diverse learners. The advent of affordable and accessible 3D printing technology makes it possible to create tactile models that represent molecules, cells, and entire organisms more accurately than traditional visual representations. We describe the assessment of a 3D gene expression puzzle as a guided inquiry learning activity in which students must correctly assemble a series of components in order to achieve an output. Upon completion of the puzzle, the TTT provides tactile feedback through vibration to signal transcriptional activation. Analysis of pre- and post-assessment performance demonstrated statistically significant increases in individual students’ paired assessment scores in two different classroom implementations, with a greater effect size at a rural minority-serving institution than an urban R1 university. These encouraging preliminary data suggest that TTTs with guided-inquiry learning disproportionately benefit disadvantaged student populations and could serve as a tool in leveling the playing field when teaching abstract biological concepts in diverse educational settings.

show abstract

Comparison of normalized gain and Cohen’sdfor analyzing gains on concept inventories

Cited by 72 publications

References 39 publications

Missing data and bias in physics education research: A case for using multiple imputation

Missing data and bias in physics education research: A case for using multiple imputation

Modernizing use of regression models in physics education research: A review of hierarchical linear modeling

Building thelacoperon: A guided-inquiry activity using 3D-printed models

Contact Info

Product

Resources

About