Data fusion with international large scale assessments: a case study using the OECD PISA and TALIS surveys

Abstract

Background: In the context of international large scale assessments, it is often not feasible to implement a complete survey of all relevant populations. For example, the OECD Program for International Student Assessment surveys both students and schools, but does not obtain information from teachers. In contrast the OECD Teaching and Learning International Survey assesses teachers and schools but does not assess students. Clearly, important information is missing from both assessments. One approach to obtaining information from both surveys is through data fusion-a variety of methods that can be used to create a synthetic data set containing information from both surveys. Methods: This paper presents an experimental evaluation of a representative group of data fusion methods using data from Iceland-the only OECD country that implemented both PISA and TALIS to all members of the relevant populations. Results: On the basis of a set of validity criterion we find that Bayesian bootstrap predictive mean matching and the EM-bootstrap methods perform best with respect to creating a usable synthetic data file for research purposes.

Factors associated with private-public school performance: Analysis of TALIS-PISA link data

“…The linked dataset combines the TALIS insights into the backgrounds, beliefs, and practices of teachers and school leaders with PISA information into backgrounds, beliefs, attitudes, and cognitive outcomes of students (Austin et al, 2015). On the contrary, in isolation, PISA only offers measures of student learning whereas TALIS only provides information about teachers' jobrelated attitudes (Kaplan and McCarty, 2013).…”

“…Few considerations make TALIS-PISA link the ideal dataset to answer our main research question. First, although in isolation PISA is missing the teacher level data and TALIS the student level data (Kaplan and McCarty, 2013), TALIS-PISA provides a linkage of the two key elements of learning outcomes within public and private schools. On the one hand, TALIS includes, among others information on how the work of teachers is recognised, reviewed, supported and rewarded and the role of school leaders (Austin et al, 2015;Sallies et al, 2016) and, on the other hand, characteristics of the students and schools are included in PISA (OECD, 2012a).…”

On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs

“…For each matrix sampling design, we implement SMI and MMI for the CQ missing data using predictive mean matching (PMM) via the R package MICE ( van Buuren and Groothuis-Oudshoorn 2010). Previous research (Kaplan and Su 2016;Kaplan and McCarty 2013) has found predictive mean matching to be quite good with respect to meeting the requirements for the validity of statistical matching and imputation set down by Räassler (2002).…”

How long is the shadow? The relationships of family background to selected adult outcomes: results from PIAAC

“…There are also well-known limitations to the policy implications of findings from cross-sectional data because of the difficulty in supporting causal interpretations of those findings. A number of authors have attempted to circumvent these difficulties either through methodologies for combining data from repeated cross-sectional surveys (Gustafsson 2013) or through linking different surveys in order to follow specific cohorts (Kaplan 2009;Kaplan and McCarty 2013;Chmielewski 2015). See the review by Chmielewski and Dhuey (2017), as well as recent work by Hampf et al (2017) and by Bind and Rubin (2017).…”