How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986–2014

Marques, Marcelo; Powell, Justin J W; Zapp, Mike; Biesta, Gert

doi:10.1177/1474904117730159

Cited by 45 publications

(58 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research, see Zapp & Powell, 2016), the UK (Teaching and Learning Research Programme, see Marques et al, 2017) and the EU's Framework Programme. Importantly, this programmification of research governance has led researchers to adapt their funding acquisition strategies (and their research, incrementally) to fit themes and expectations espoused by funding bodies.…”

Section: Discussionmentioning

confidence: 99%

(Re)shaping educational research through ‘programmification’: Institutional expansion, change, and translation in Norway

Zapp

Helgetun

Powell

2018

Euro J of Education

Self Cite

View full text Add to dashboard Cite

Educational research in Norway has experienced unprecedented structural expansion and cognitive shifts over the last two decades because of greater state investments and the strategic use of extensive and multi-year thematic programmes to fund research projects.Using a neo-institutionalist framework, we examine institutionalisation dynamics in cultural-cognitive, normative and regulative dimensions by means of interviews, research programme calls, policy documents and funding data. In the cultural-cognitive dimension, we find references to the knowledge society, the perceived importance of evidence in policymaking and ideas of quality, excellence and relevance. In the normative dimension, we find the introduction of new professional and methodological standards, reflecting broader global patterns of academic and epistemic drift. In the regulative dimension, the strengthened role of both government and the Research Council of Norway is manifest in substantial growth in both funding and large-scale, long-term planning, including thematic choices-evidence of 'programmification'. The importance of external models has grown in an era of internationalisation, yet translation occurs at every level of the governance of educational research. This results in a specific Norwegian research model, guided by a mode of governance of programmes that maintains social values that are traditionally strong in Nordic societies.

show abstract

Section: Discussionmentioning

confidence: 99%

(Re)shaping educational research through ‘programmification’: Institutional expansion, change, and translation in Norway

Zapp

Helgetun

Powell

2018

Euro J of Education

Self Cite

View full text Add to dashboard Cite

show abstract

“…Evidence for this increase comes from regular government accountability exercises for evaluating the quality of research (e.g. Higher Education Funding Council for England, 2008England, , 2015, and some authors have argued that these exercises are also the drivers for the increase (Gilroy & McNamara, 2009;Marques et al, 2017). Similar exercises are increasingly common around the world (Hicks, 2012), some of which are influenced by the UK model (e.g.…”

Section: Introductionmentioning

confidence: 99%

Measuring conceptual understanding in randomised controlled trials: Can comparative judgement help?

Jones

Bisson

Gilmore

et al. 2019

British Educational Res J

View full text Add to dashboard Cite

An impediment to conducting high‐quality quantitative research studies in education is the paucity of valid measures of learning gains. Studies often seek to investigate students’ deep, conceptual understanding yet many measures assess only surface, procedural understanding. One reason is that the development of validated measures of conceptual understanding is resource intensive and time consuming, and success is not guaranteed. We evaluated a novel and efficient technique, based on comparative judgement, for assessing conceptual understanding. We applied the technique to a randomised controlled trial in which students were taught simple algebra based on either the Grid Algebra or the MiGen software package. The participants were Year 5 students (N = 188) drawn from four primary schools who had not encountered algebra previously. An instrument from the literature (Concepts in Secondary Mathematics and Science: Algebra Scale) and a novel comparative judgement assessment were administered following the intervention. Students in the Grid Algebra condition outperformed those in the MiGen condition on both post‐test measures. The comparative judgement technique performed similarly to the standard instrument but was far more efficient to design and implement. The technique can, in principle, be quickly applied to any target concept of interest. We conclude that comparative judgement is a valid, reliable and practical tool that could help to increase both the quantity and quality of quantitative research in education.

show abstract

“…In an era of ubiquitous evaluation, R&D governance is based on explicitly measured and evaluated outputs, mainly research articles, less the scholarly monographs that provide the in-depth contextualisation so crucial for fundamental research on education systems, cultures, and practices. Indeed, increased emphasis on research articles is an unintended consequence of research evaluation (Marques et al 2017). The effects and changed behaviour of individuals and organisations in such regimes has garnered attention, yet few longitudinal studies uncover long-term trends in largely university-based research (but see Fernandez [2017] 2019).…”

Section: Comparisons Governance Evaluation and Supranational Coordmentioning

confidence: 99%

“…Yet attempts to transform qualities into numeric forms to measure, compare, and inform decision-making reduce the quality of information. This threatens to narrow the recognition and impact of knowledge generated in diverse systems, especially as media companies convert peer review judgements into products to market for profit, such as higher education rankings (Marques et al 2017;Marques and Powell 2019). Further discipline-specific studies are needed to emphasise how research evaluation impacts the structural organisation and cognitive development of research, reinforcing stratification and standardisation.…”

Section: Comparisons Governance Evaluation and Supranational Coordmentioning

confidence: 99%

Comparative education in an age of competition and collaboration

Powell

2020

Comparative Education

View full text Add to dashboard Cite

Comparative education relies on experiences, expertise, data, and criticism derived from multiple contexts and diverse levels to generate insights, facilitate understanding, and explain change. Marked by connectivity, our contemporary era vastly increases the (potential) diffusion of ideas essential for scientific advance. Three interlocking trends emphasise the growing relevance of comparative educational research. Firstly, competition has become more potentamong scholars, their organisations, and within as across countries. Secondly, educational studies, as science more generally, are increasingly conducted in collaboration -across disciplinary, cultural, linguistic, and organisational boundariesenhancing the potential for discovery while producing influential scholarship. Thirdly, while educational research and policymaking are increasingly comparative, comparative knowledge stores are often only selectively used. To counter such reductionism, indepth comparative institutional analysis across divides of academy, politics, and practice remain crucial. The multidisciplinary field must claim its relevance more persuasively, even as scholarly exchange, mobilities, and cultural knowledge endure as vital foundations. Introduction: comparative education between experience, exchange and evaluationComparative education relies on experiences, expertise, data, and criticism derived from multiple contexts and diverse levels to generate insights, facilitate understanding, and explain change. We live in an age marked by extraordinary mobility, an encompassing Internet, and English as the increasingly-dominant scientific lingua franca. These phenomena extend our connectivity and vastly increase the (potential) diffusion of ideasthe essence of scientific advanceand policy learning, as challenging as it remains to adapt solutions found within complex institutional settings to others. Worldwide, institutions and organisations of education and science have dramatically expanded, becoming key sites of exchange and debate that are crucial for innovation. From international conference participation and academic exchange to sabbaticals and even careers abroad, individual spatial mobility has become a sine qua non of the (successful) scientific career (see Kim 2017). For comparative and international education, even more for

show abstract

How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986–2014

Cited by 45 publications

References 53 publications

(Re)shaping educational research through ‘programmification’: Institutional expansion, change, and translation in Norway

(Re)shaping educational research through ‘programmification’: Institutional expansion, change, and translation in Norway

Measuring conceptual understanding in randomised controlled trials: Can comparative judgement help?

Comparative education in an age of competition and collaboration

Contact Info

Product

Resources

About