REF could leave universities with 'huge' bill, expert warns

Australian model shows that institutions bear data-gathering burden. Zoe Corbyn reports

May 8, 2008

Universities could face "huge" costs managing the system that will replace the research assessment exercise, an expert has warned.

The research excellence framework was designed to help "reduce the administrative burden on universities" by replacing the RAE's peer review with a system that makes greater use of statistical indicators of research quality, such as data on the number of times an academic's published research is cited.

The Higher Education Funding Council for England is moving into a pilot stage for the REF, during which it will decide which data are to be collected, which databases will be used and how much information individual institutions will have to provide, analyse and verify.

But in a conference speech last week, Linda Butler, head of the Research Evaluation and Policy Project at the Australian National University, said that when a similar metrics system was implemented in Australia in the 1990s, universities bore most of the data-collection burden.

"(The particular methodology chosen) is going to have direct implications on how much it is going to cost each institution and Hefce itself - not just in terms of purchasing data ... but (also) the implementation costs and staff time," Ms Butler told the conference, "Beyond the RAE 2008: Bibliometrics, League Tables and the REF".

Another conference participant said that even if Hefce ultimately takes responsibility for data-gathering in the REF, individual universities will still undertake parallel exercises to ensure that they are not missing out on funding.

Ms Butler said that the development of institutional information systems is complex and takes time, and the "choice between simplicity and complexity has huge implications for the costs of the exercise".

Ms Butler warned that vice-chancellors lobbying for particular research evaluation systems based on what they believed would benefit their universities often had "no understanding" of the cost involved.

A pre-conference survey of delegates showed that their biggest worry about the new metrics system was "getting accurate and verifiable data" - 28 per cent described this as their main concern. About 20 per cent said their main fear centred on the "time and workload" implications.

Ms Butler's concerns were reiterated by Jonathan Adams, director of the data analysis firm Evidence Ltd, who said: "The cost will almost certainly be larger than we are expecting." He also expressed doubts about how long metrics could accurately reflect research excellence: "As soon as you start to use an indicator for public policy, it will lose the information value that made it a good metric in the first place."

Graeme Rosenberg, Hefce's REF pilot manager, told the conference that the sector accepted that a way of making a metrics system work could be found.

zoe.corbyn@tsleducation.com

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Most Commented

James Fryer illustration (27 July 2017)

It is not Luddism to be cautious about destroying an academic publishing industry that has served us well, says Marilyn Deegan

Hand squeezing stress ball
Working 55 hours per week, the loss of research periods, slashed pensions, increased bureaucracy, tiny budgets and declining standards have finally forced Michael Edwards out
Jeffrey Beall, associate professor and librarian at the University of Colorado Denver

Creator of controversial predatory journals blacklist says some peers are failing to warn of dangers of disreputable publishers

Kayaker and jet skiiers

Nazima Kadir’s social circle reveals a range of alternative careers for would-be scholars, and often with better rewards than academia

hole in ground

‘Drastic action’ required to fix multibillion-pound shortfall in Universities Superannuation Scheme, expert warns