REF may cause metrics 'creep', analyst warns

May 8, 2008

The use of statistical indicators to judge research quality is likely to "creep" into the social sciences under the system being set up to replace the research assessment exercise, a sector analyst warned this week.

The Government confirmed last month that under the forthcoming research excellence framework, research quality in all subjects will be judged using a combination of metrics, such as data on the number of times an academic's work is cited by other academics, and forms of peer review.

The weighting of these two elements would vary according to the subject area, with a greater use of metrics expected in the science subjects, where it is more established and accepted.

But speaking last week at a conference, "Beyond the RAE 2008: Bibliometrics, League Tables and the REF", Jonathan Adams, the director of the data analysis firm Evidence Ltd, warned there was a heightened potential under the REF for metrics to be incorporated into some non-science subjects.

"What worries me a little bit about the recent announcement that metrics will be applied - along with peer review - to the social sciences is that this is going to be metrics creep," he told conference attendees.

"What we are now going to see is really that the barriers are being removed and ... gradually the metrics process rolling forward inexorably."

Dr Adams said the key to making the REF work would be for academics in the individual disciplines - along with their professional bodies and universities - to decide what the outcome should be to ensure that the right balance was being applied.

"In the end it is the community that absolutely must engage with this (process)," he said. "They really need to evaluate this and see what the different packages are that make sense in different places."

Dr Adams also used the conference to announce the forthcoming launch of "You are the REF", a tool that will allow academic departments to simulate how they might fare under a metrics system. It is to be made available shortly at http://www.rae2008.com.

zoe.corbyn@tsleducation.com.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Most Commented

James Fryer illustration (27 July 2017)

It is not Luddism to be cautious about destroying an academic publishing industry that has served us well, says Marilyn Deegan

Jeffrey Beall, associate professor and librarian at the University of Colorado Denver

Creator of controversial predatory journals blacklist says some peers are failing to warn of dangers of disreputable publishers

Hand squeezing stress ball
Working 55 hours per week, the loss of research periods, slashed pensions, increased bureaucracy, tiny budgets and declining standards have finally forced Michael Edwards out
Kayaker and jet skiiers

Nazima Kadir’s social circle reveals a range of alternative careers for would-be scholars, and often with better rewards than academia

hole in ground

‘Drastic action’ required to fix multibillion-pound shortfall in Universities Superannuation Scheme, expert warns