Hefce backs off citations in favour of peer review in REF

Research assessments in hard sciences will now be 'informed' by bibliometrics. Zoe Corbyn writes

June 18, 2009

The use of citations to determine the quality of academic work in the hard sciences is to be abandoned in favour of peer review in the new system being designed to replace the research assessment exercise.

However, information about the number of citations a scholar's work accrues could be provided to assessment panels to help "inform" their judgments in a range of subjects.

At a conference on the forthcoming Research Excellence Framework (REF) this week, the Higher Education Funding Council for England sketched out how it intends to assess the quality of research outputs in the system, which will determine the allocation of £1.6 billion of annual research funding from 2014.

"We just don't think bibliometrics are sufficiently mature at this stage to be used in a formulaic way or, indeed, to replace expert review," said Graeme Rosenberg, Hefce's REF project manager. "However, there is still scope for bibliometrics to inform the assessment process."

The announcement follows an interim report on the REF bibliometrics pilot exercise.

Although all decisions are yet to go to the Hefce board ahead of a consultation paper on the new system due later this year, the council told about 300 conference delegates that it was effectively sidelining the use of citations - a measurement of the number of times academics' work is cited by their peers - as the main determinant of research quality in the remaining hard science subjects for which it thought their use was possible. Instead, it is to draw up a list of subjects where it is appropriate for citations to inform peer review.

Dr Rosenberg also outlined Hefce's preferred model for assessing research quality. As in the RAE, institutions would continue to select which staff and which of up to four of their outputs are submitted.

The first exercise would cover work published between 2008 and 2012, and data on citations accrued for the papers in the same period would also go to panels.

"Panels might look at (citations for) each paper or (citations for) the whole submission. We are still working through the details," Dr Rosenberg told Times Higher Education.

Putting citation data for each paper before panels would be a radical departure from earlier plans, which until now have mooted citation profiles for institutions only in broad subject areas.

Charles Oppenheim, professor of information science at Loughborough University, attended the conference. He said it was now likely that pressure on scholars to increase their citations would grow.

"The new game to play is citations. One of the games, without any doubt, is that senior staff will encourage academics to cite within the institution," he said.

Dr Rosenberg also discussed how Hefce intended to assess the social and economic impact of research. The process would take place at a departmental level and would be based on a narrative backed by indicators and exemplars, he said.

The first REF would assess impact made between 2008 and 2012, he said, although the research itself could have been carried out earlier.

zoe.corbyn@tsleducation.com.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

Female professor

New data show proportion of professors who are women has declined at some institutions

celebrate, cheer, tef results

Emilie Murphy calls on those who challenged the teaching excellence framework methodology in the past to stop sharing their university ratings with pride