Please cite as the following: Walkington, C., Clinton, V., & Shivraj, P. (2018). How readability factors are differentially associated with performance for students of different backgrounds when solving math word problems.
AbstractThe link between reading and mathematics achievement is well known, and an important question is whether readability factors in mathematics problems are differentially impacting student groups. Using 20 years of data from the National Assessment of Educational Progress and the Trends in International Mathematics and Science Study, we examine how readability factors -such as length, word difficulty, and pronouns -interact with student background characteristics -such as race/ethnicity, mathematics achievement, and socioeconomic status.Textual features that make problems more difficult to process appear to differentially negatively impact struggling students, while features that make language easier to process appear to differentially positively impact struggling students. It is critical that readability along various dimensions be considered when designing instruction and assessment.
INTRODUCTION:
As part of continuing certification, the American Board of Obstetrics and Gynecology (ABOG) diplomates complete article-based assessments annually. Diplomates read articles, respond to article questions, and must score above a certain threshold to demonstrate competence in their understanding. There is a paucity of evidence that diplomates retain the information after reading articles. The study objective was to evaluate the effectiveness of the two question types (understanding and application) at measuring diplomates’ retention of knowledge over time.
METHODS:
A sample of 344 diplomates who chose selected articles between January-March 2019 volunteered for the study. Diplomates completed article questions in a business-as-usual setting (with article access) and were assessed again during summer 2019 (without article access). Each participant received 2 questions (one of each type) from 5 of the selected articles, for a total of 10 questions. The timed summer assessment was used as a proxy for retention. Diplomates’ scores, by item type, from each assessment setting were compared using a t-test to determine whether information was retained from the articles.
RESULTS:
The information assessed by application items was retained at a significantly higher rate (80.58%) than understanding items (60.75%), as demonstrated by mean scores on the assessment (P<.05).
CONCLUSION:
Application items yielded a higher knowledge retention rate. Understanding items ensure diplomates comprehend the article while application questions ensure diplomates can generalize and apply their understanding from the article. A combination of both item types for article-based assessments is necessary and this study demonstrates that diplomates retain the information they read from articles.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.