Large-scale assessments in education, such as IEA's TIMSS and PIRLS, rely on sophisticated assessment instruments, elaborate sample designs, and leading-edge item response theory to meet their analytical objectives. Both assessments provide a rich and complex database intended to support and promote secondary analyses. This paper describes the complex international database structures and the statistical methods and procedures for analyzing TIMSS and PIRLS data, with examples provided for illustration. Three essential elements must be considered by researchers in any statistical analysis based on data from the TIMSS and PIRLS international databases. The first is the use of sampling weights in order to produce accurate and reliable results. Second, both assessments apply the Jackknife Repeated Replication technique to derive proper estimates of sampling variance. Finally, with student achievement reported as sets of five plausible values, statistical analyses are performed five times, once for each plausible value, and the final results aggregated across the five plausible values. Researchers and users of the TIMSS and PIRLS international databases who conduct their analyses as described in this paper should feel confident in the results their analyses will yield.

No abstract

No abstract

The article will describe the opportunities that TIMSS data provide for evidence-based decision making. TIMSS collects extensive data about the mathematics and science curricula in the participating countries, as well as the characteristics of effective schools and classroom environments for learning. Participating countries use the results in various ways to explore educational issues, and several examples will be described.

Background: TIMSS 2019 is the first assessment in the TIMSS transition to a computerbased assessment system, called eTIMSS. The TIMSS 2019 Item Equivalence Study was conducted in advance of the field test in 2017 to examine the potential for mode effects on the psychometric behavior of the TIMSS mathematics and science trend items induced by the change to computer-based administration. Methods: The study employed a counterbalanced, within-subjects design to investigate the potential for eTIMSS mode effects. Sample sizes for analysis included 16,894 fourth grade students from 24 countries and 9,164 eighth grade students from 11 countries. Following a review of the differences of the trend items in paper and digital formats, item statistics were examined item by item and aggregated by subject for paperTIMSS and eTIMSS. Then, the TIMSS scaling methods were applied to produce achievement scale scores for each mode. These were used to estimate the expected magnitude of the mode effects on student achievement. Results: The results of the study provide support that the mathematics and science constructs assessed by the trend items were mostly unaffected in the transition to eTIMSS at both grades. However, there was an overall mode effect, where items were more difficult for students in digital formats compared to paper. The effect was larger in mathematics than science. Conclusions: Because the trend items cannot be expected to be sufficiently equivalent across paperTIMSS and eTIMSS, it was concluded that modifications must be made to the usual item calibration model for TIMSS 2019 to measure trends. Each eTIMSS 2019 trend country will administer paper trend booklets to a nationally representative sample of students, in addition to the usual student sample, to provide a bridge between paperTIMSS and eTIMSS results.