Item readability and science achievement in TIMSS 2003 in South Africa

Abstract

This study investigated the relationship between readability of 73 text-only multiple-choice questions from Trends in International Mathematics and Science Study (TIMSS) 2003 and performance of two groups of South African learners: those with limited English-language proficiency (learners attending African schools) and those with better English-language proficiency (learners attending non-African schools). Both groups were exposed to the same intended curriculum, but differed with respect to the quality of teaching they received, the availability of resources, and the level of functionality of their schools. Learners from non-African schools performed significantly better than learners from African schools. Three readability factors (sentence complexity, unfamiliar words, and long words) were analyzed. High sentence complexity resulted in random guessing in non-African schools, and favoring an incorrect answer in African schools. Some TIMSS items have complex wording, with numerous prepositional phrases and clauses, and unclear questions. Recommendations for maximum readability and comprehensibility were not met, and these items are therefore invalid for learners with limited English-language proficiency. Learners employ a range of strategies in attempting to answer questions that they do not understand. Overall, though, poor readability of TIMSS items does not fully account for South African learners' poor performance in TIMSS.

Development of a model of effectiveness in science education to explore differential science performance: A case of South Africa

“…This phenomenon is understandable as students from minority ethnic groups have to learn science knowledge in an instruction language that is different from mother tongue (Rollnick, 2000). There are 11 official languages in South Africa and the poor command in the language of instruction complicates conceptual understanding and affects science performance negatively (Dempster & Reddy, 2007). It is evident that students who are not familiar with the language of instruction cannot understand content taught in class.…”

Ideas about the Human Body among Secondary Students in South Africa

“…The other two (42 and 19 students) have no laboratory facilities; the classroom is equipped with only chairs, tables and a blackboard. It has been shown that TIMSS questions are complicated for students who are not home-language English speakers (Dempster & Reddy, 2007). Hence difficulties related to language were anticipated in the last two schools, and instructions and questions were translated by a translator during the data collection.…”

“…However, very few studies have been done in Africa. South African students perform particularly poorly compared with European countries in TIMSS (Trends in International Mathematics and Science Study) studies, indicating poor subject knowledge of science (Dempster & Reddy, 2007). It can reasonably be predicted that South African students would be less knowledgeable about the structure and functions of human organ systems and the interconnections between organ systems than students in European countries.…”

Mitigating the effect of language in the assessment of science: A study of English‐language learners in primary classrooms in the United Kingdom

“…Shaw () discusses how different foci may explain variations in ELLs achievement, suggesting performance on tasks requiring dependence on text is more significantly affected by language than those requiring graphs or calculation of formulae. Furthermore, Dempster and Reddy () investigated how sentence complexity, specifically the use of unfamiliar and long words differentially affects ELL and ENS learners’ performance on multiple‐choice questions in Mathematics and Science leading to poorer performance for learners who had limited English‐language proficiency. However, while there is considerable research in the language teaching, pedagogy, and assessment fields with regard with the impact of task type on learner performance, we were unable to locate research focusing on the specific types of knowledge (scientific and linguistic) that we believe comprise most forms of Science assessment tasks at primary school level between the ages of 7 and 11.…”