REF alternative delivers similar outcome for QR funding

Report says distribution would change little if based on institutions’ research council income

April 11, 2013

Source: Alamy

Like pulling teeth? Some say costly REF measurements should be scrapped

Allocating an institution’s quality-related research funding on the basis of its research council income would not significantly change outcomes, a government-commissioned report has concluded.

The report - by the Centre for Business Research, based at the University of Cambridge, and the UK Innovation Research Centre, based at Cambridge and Imperial College London - examines the effect of the UK’s dual funding system for research on knowledge exchange and commercialisation.

It notes a “strong” correlation between the amount of research income that institutions earn from the various sources.

Some observers have long argued that the costly and labour-intensive research assessment exercise, and its successor, the research excellence framework, should be scrapped and QR funding allocated based on each institution’s research council income.

The report, Dual Funding Structure for Research in the UK: Research Council and Funding Council Allocation Methods and the Pathways to Impact of UK Academics, notes that the 2008 RAE’s cost to the sector has been estimated at nearly £50 million.

The researchers examined what would have happened if mainstream QR funding in 2009-10 and 2010-11 had been distributed on the basis of each institution’s share of research council income earned between 2002-03 and 2008-09. They found no average difference between the actual and hypothetical allocations.

The QR allocation of the biggest losers would have slipped by 0.9 per cent, while that of the biggest winners would have risen by 2.6 per cent. Most of the winners would have been the “top handful” of institutions because of greater concentration in research council funding.

In 2010, the top 10 per cent of institutions claimed 53 per cent of QR income and 64 per cent of research council income. However, from 2012-13 QR funding became more concentrated after 2* research was removed from the formula. As a result, the mainstream QR allocations of England’s five biggest earners - Imperial College, University College London and the universities of Oxford, Cambridge and Manchester - slipped by 0.9 per cent, compared with an overall drop of 3.2 per cent.

But Mark Burnley, senior lecturer in the University of Kent’s School of Sport and Exercise Sciences, said that the study’s statistical interpretation and presentation of data - such as its lack of correlation plots and error bars - was “so poor as to render the information almost meaningless”.

He said that using research council income to determine the allocation of QR funding would be “fundamentally unfair” because it would work against disciplines that derived higher proportions of their income from other sources. It would also disadvantage newer institutions or departments that had not yet won any research council income.

Giles Carden, director of strategic planning and analytics at the University of Warwick, said a previous Treasury study had also shown a high correlation between research council income and RAE scores. Allocating QR cash on the basis of the former made sense “if the government is simply interested in [finding] the simplest method possible”, he said. But, he continued, the REF also provided a means to assess the quality of the outputs of UK research and had spurred universities to focus on maximising their research excellence.

paul.jump@tsleducation.com

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (1)

Equally, there is a statistically significant correlation between previous RAE scores and numbers of citations received by Departments for their outputs over the relevant period. I remain intrigued that so many people are opposed to simpler, cheaper ways of measuring research excellence.

Have your say

Log in or register to post comments

Featured Jobs

Reader in Politics and Policy

St Marys University, Twickenham

Engineer

Cern

Professor of Anthropology

Maynooth University

Preceptor in Statistics

Harvard University

Postdoctoral Fellowship in Electrochemistry

Norwegian University Of Science & Technology -ntnu
See all jobs

Most Commented

Doctoral study can seem like a 24-7 endeavour, but don't ignore these other opportunities, advise Robert MacIntosh and Kevin O'Gorman

Matthew Brazier illustration (9 February 2017)

How do you defeat Nazis and liars? Focus on the people in earshot, says eminent Holocaust scholar Deborah Lipstadt

Improvement, performance, rankings, success

Phil Baty sets out why the World University Rankings are here to stay – and why that's a good thing

Laurel and Hardy sawing a plank of wood

Working with other academics can be tricky so follow some key rules, say Kevin O'Gorman and Robert MacIntosh

Warwick vice-chancellor Stuart Croft on why his university reluctantly joined the ‘flawed’ teaching excellence framework