Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates

Abstract

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

Tools for Change: Measuring Student Conceptual Understanding Across Undergraduate Biology Programs Using Bio-MAPS Assessments

“…These time points typically include beginning of introductory biology : the start of the program (e.g., prior to classes or at the beginning of the first course in the major), end of introductory biology : after the completion of an introductory course series, and advanced : at the end of courses typically taken by seniors just prior to graduation. Within our multi-institution data sets, we find overall increases in student performance over time for each Bio-MAPS instrument (12, 14, 15). Students demonstrated large gains for some concepts, while other concepts remained challenging, even for students about to graduate.…”

“…To help alleviate issues associated with designing program assessments, we developed a suite of freely available instruments called Biology-Measuring Achievement and Progression in Science or Bio-MAPS. These assessments align with the core concepts from Vision and Change (6, 7), with separate instruments for general biology (12), molecular biology (13), physiology (14), and ecology/evolution (15). We followed established methods of assessment development (16), including optimizing response validity through student interviews, ensuring technical accuracy through expert reviews, pilot testing at a variety of institutions across the country, and addressing any potential item biases favoring particular demographic groups.…”

Resources for Teaching and Assessing the Vision and Change Biology Core Concepts

“…Each of these assessment tools followed a similar set of design principles and is presented to students as restricted-response, multiple T/F, or likely-unlikely items designed to assess the conceptual understanding of a large number of students outside class. There are four separate instruments: general biology (GenBio-MAPS; Couch et al, 2019), molecular biology (Molecular Biology Capstone Assessment, MCBA; Couch et al, 2015), physiology (Phys-MAPS; Semsar et al, 2019), and ecology/evolution (EcoEvo-MAPS; Summers et al, 2018). All of the Bio-MAPS instruments are freely available and can be administered through an online web portal at http://cperl.lassp.cornell.edu/bio-maps (Smith et al, 2019).…”

“…The instruments can also be used to measure progress at 2-year colleges by administering them at the beginning and end of an introductory biology series or at the beginning and end of a specialized set of courses (e.g., anatomy and physiology). Data suggest that students show greater understanding of the core concepts at more advanced levels compared with introductory levels (Summers et al, 2018;Couch et al, 2019;Semsar et al, 2019). Student performance on these assessments could be used to help departments identify the extent to which students learn the core concepts in introductory or upper-level courses and help departments assess their proficiency in teaching the core concepts over the duration of a degree.…”

Using the Ecology and Evolution‐Measuring Achievement and Progression in Science assessment to measure student thinking across the Four‐Dimensional Ecology Education framework

“…), and physiology (Semsar et al. ). To use this portal, instructors fill out a brief survey (average completion time is 4.3 min) and then receive a web link that they share with their students.…”