Work in Progress: Validity and Reliability Testing of the Engineering Concept Assessment Modified for Eighth Grade

Abstract

She holds a Ph.D. in Educational Research and Measurement and M.E. in Educational Psychology. Dr. Koskey teaches courses in evaluation, assessment, research design, and statistics. She also works as a psychometric consultant and serves on the Editorial Board for the journal of Psychological Assessment. Her work is published in leading journals such as Studies in Educational Evaluation, Journal of Applied Measurement, Journal of Mixed Methods Research, Journal of Experimental Education, International Journal of Qualitative Methods, and Educational and Psychological Measurement. Further, she has authored book chapters on Norming and Scaling for Automated Essay Scoring and Data-driven STEM Assessment. Dr. Koskey has secured grant funding from the Ohio Department of Education and National Science Foundation (NSF), as well as contributed to the evaluations on grants funded by the ODE, U.S. Department of State, and NSF.

Modification and validation of the mixed‐format Engineering Concept Assessment for middle school students using many‐facet Rasch measurement

“…Original authors of the ECA provided written permission for the authors to administer the 20-item measure that consisted of 13 MC, one short answer, and six CR items. This assessment was modified for the eighth-grade level, resulting in the ECA Modified for 8 th Grade (ECA M-8; Koskey, Garafolo, Makki, Ahmed, Visco, & Samreddy, 2017). Koskey et al (2017) detailed an iterative modification and development process that utilized qualitative and quantitative methods for informing revisions, consistent with the Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014).…”

Board 154: Zipping to STEM: Integrating Engineering Design in Middle School Science

“…Assessment [8] was used to assess students' understanding of engineering design. This modified form, ECA-M8 [9] was aligned to the eighth-grade state standards including the Next Generation Science Standards [6]. The ECA-M8 consisted of 13 multiple-choice items assessing basic understanding of engineering design concepts and one design problem testing the ability to transfer the concepts to a new design problem not previously presented as part of the learning unit.…”

“…The students' unadjusted and adjusted mean ECA-M8 [8,9] scores on the pre-test and post-test are illustrated in Figure 1 and reported in Table 3.…”

Assessing the Reliability of a Chemical Engineering Problem-solving Rubric when Using Multiple Raters

“…Methods of student assessment often incorporate rater-mediated assessment [10][11][12][13]. These methods of assessing student knowledge move beyond traditional notions of student grades that are just the calculation of correct responses divided by total possible items on a formative test.…”