Using League Table Rankings in Public Policy Formation: Statistical Issues

Goldstein, Harvey

doi:10.1146/annurev-statistics-022513-115615

Cited by 14 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Importantly, when multidimensionality interacts with this systematic sampling bias, rank orderings can be reversed. Goldstein (2014) states "I am not suggesting that league tables should never be published. Quantitative data that bear on performance are a useful tool for addressing the clear need for accountability from public (and other) institutions.…”

Section: Discussionmentioning

confidence: 99%

The structure of surveys and the peril of panels

Simpson

2016

Studies in Higher Education

View full text Add to dashboard Cite

University league tables give the image that there is a single dimension along which institutions can be placed. Most derive rankings from an aggregate score of multiple items, which often include opinion responses which has the potential to introduce sampling bias. This paper explores what happens when the two issues of dimensionality and sampling bias interact in league tables. It uses the Times Higher Education Student Experience Surveywhich uses a panel design-to explore these issues. It notes that they combine to produce a distorted image of the relative quality of the student experience in different institutions. We conclude that ignoring dimensionality and the systematically unrepresentative nature of the sample could lead policy makers to draw inappropriate conclusions.

show abstract

Section: Discussionmentioning

confidence: 99%

The structure of surveys and the peril of panels

Simpson

2016

Studies in Higher Education

View full text Add to dashboard Cite

show abstract

“…The assessment of learning and student outcomes in education has a long history in England (Leckie & Goldstein, 2009. The English education system has relied heavily on measurement of student learning through exams (Goldstein & Leckie, 2016) and the usage of these data for accountability and policy-making purposes (Department for Education, 2016, 2017aGoldstein, 2014). In relation to higher education, however, it is the US (Pascarella & Terenzini, 1991, Australia (e.g.…”

Section: Literature Reviewmentioning

confidence: 99%

Building the foundations for measuring learning gain in higher education: a conceptual framework and measurement instrument

Vermunt

Ilie

Vignoles

2018

Higher Education Pedagogies

View full text Add to dashboard Cite

In this paper, we set out the first step towards the measurement of learning gain in higher education by putting forward a conceptual framework for understanding learning gain that is relevant across disciplines. We then introduce the operationalisation of this conceptual framework into a new set of measurement tools. With the use of data from a large-scale survey of 11 English universities and over 4,500 students, we test the reliability and validity of the measurement instrument empirically. We find support in the data for the reliability of most of the measurement scales we put forward, as well as for the validity of the conceptual framework. Based on these results, we reflect on the conceptual framework and associated measurement tools in the context of at-scale deployment and the potential implications for policy and practice in higher education. ARTICLE HISTORY

show abstract

“…On the one hand, these weights guarantee that the numbers, for instance the top10% values, keep constant across various fields, on the other hand, such weights might be difficult in a statistical analysis (e.g., van den Boogart & Tolosana-Delgado, 2013). What is lacking is an overarching statistical model of the data of the sort that mathematicians and statisticians have called for (Adler, Ewing, & Taylor, 2009;Goldstein, 2014).…”

Section: What Are the Effects Of Weightings At The Level Of Individuamentioning

confidence: 99%