Study Design and Evolution, and the Imperatives of Reliability and Validity

Abstract

During the six decades since the International Association for the Evaluation of Educational Achievement (IEA) began its program of studies, an ever-growing political impetus worldwide for improved educational provision has stimulated countries' willingness to participate in international large-scale comparative assessments of learning outcomes. However, research within the complex multinational context that IEA operates in has resulted in significant methodological and technological challenges for the researchers and practitioners endeavoring to meet the goals of IEA studies. Such studies must satisfy the twin imperatives of validity and reliability, rarely an easy task given the multiple and diverse cultures, languages, scripts, educational structures, educational histories, and traditions of the countries and jurisdictions that participate. An appreciation of IEA's underlying assessment philosophy is fundamental to understanding the Association's assessment goals and the key design features of its studies, and what these mean with respect to ensuring that the studies satisfactorily address the demands of reliability and validity.

Examining the Predictive Capacity of School-based Assessment on Students' Mathematics Performance in Large-scale Examinations

“…According to Hernández-Torrano, and Courtney, [9] these assessments often take the form of standardized tests, surveys, or examinations administered to a representative sample of students within a particular educational jurisdiction. The primary goal of large-scale assessments in education is to provide policymakers, educators, and stakeholders with valuable information about the effectiveness of educational systems, identify areas for improvement, monitor trends in student achievement over time, and make data-driven decisions to enhance teaching and learning practices [10]. Examples of large-scale assessments in education include national standardized tests like the National Assessment of Educational Progress (NAEP) in the United States or international assessments like the Programme for International Student Assessment (PISA) conducted by the Organisation for Economic Co-operation and Development (OECD).…”