Hefce backs off citations in favour of peer review in REF

Research assessments in hard sciences will now be 'informed' by bibliometrics. Zoe Corbyn writes

June 18, 2009

The use of citations to determine the quality of academic work in the hard sciences is to be abandoned in favour of peer review in the new system being designed to replace the research assessment exercise.

However, information about the number of citations a scholar's work accrues could be provided to assessment panels to help "inform" their judgments in a range of subjects.

At a conference on the forthcoming Research Excellence Framework (REF) this week, the Higher Education Funding Council for England sketched out how it intends to assess the quality of research outputs in the system, which will determine the allocation of £1.6 billion of annual research funding from 2014.

"We just don't think bibliometrics are sufficiently mature at this stage to be used in a formulaic way or, indeed, to replace expert review," said Graeme Rosenberg, Hefce's REF project manager. "However, there is still scope for bibliometrics to inform the assessment process."

The announcement follows an interim report on the REF bibliometrics pilot exercise.

Although all decisions are yet to go to the Hefce board ahead of a consultation paper on the new system due later this year, the council told about 300 conference delegates that it was effectively sidelining the use of citations - a measurement of the number of times academics' work is cited by their peers - as the main determinant of research quality in the remaining hard science subjects for which it thought their use was possible. Instead, it is to draw up a list of subjects where it is appropriate for citations to inform peer review.

Dr Rosenberg also outlined Hefce's preferred model for assessing research quality. As in the RAE, institutions would continue to select which staff and which of up to four of their outputs are submitted.

The first exercise would cover work published between 2008 and 2012, and data on citations accrued for the papers in the same period would also go to panels.

"Panels might look at (citations for) each paper or (citations for) the whole submission. We are still working through the details," Dr Rosenberg told Times Higher Education.

Putting citation data for each paper before panels would be a radical departure from earlier plans, which until now have mooted citation profiles for institutions only in broad subject areas.

Charles Oppenheim, professor of information science at Loughborough University, attended the conference. He said it was now likely that pressure on scholars to increase their citations would grow.

"The new game to play is citations. One of the games, without any doubt, is that senior staff will encourage academics to cite within the institution," he said.

Dr Rosenberg also discussed how Hefce intended to assess the social and economic impact of research. The process would take place at a departmental level and would be based on a narrative backed by indicators and exemplars, he said.

The first REF would assess impact made between 2008 and 2012, he said, although the research itself could have been carried out earlier.

zoe.corbyn@tsleducation.com.

Already registered?

Sign in now if you are already registered or a current subscriber. Or subscribe for unrestricted access to our digital editions and iPad and iPhone app.

Have your say

Log in or register to post comments

Register to continue  

You've enjoyed reading five THE articles this month. Register now to get five more, or subscribe for unrestricted access.

Most Commented

Track runner slow off the starting blocks

Lack of independent working blamed for difficulties making the leap from undergraduate to doctoral work

Quality under magnifying glass

Hefce's new standards regime will enable universities to focus on what matters to students, says Susan Lapworth

Woman tearing up I can't sign

Schools and universities are increasingly looking at how improving personalities can boost social mobility. But in doing so, they may be forced to choose between teaching what is helpful, and what is true, says David Matthews

Door peephole painted as bomb ready to explode

It’s time to use technology to detect potential threats and worry less about outdated ideas of privacy, says Ron Iphofen

A keyboard with a 'donate' key

Richard Budd mulls the logic of giving money to your alma mater