REF panellists must be absolutely clear: quality in the UK is not in decline

Research excellence framework panels must be sure to make absolute rather than relative judgements of submissions to avoid the impression that the quality of the UK research base is in decline, a pro vice-chancellor has warned.

December 1, 2011

Universities' expectation that future funding formulas will take into account only work rated 3* ("internationally excellent") and 4* ("world-leading") means that many are planning to submit only those outputs they believe fall into these categories to the 2014 REF.

But Adam Tickell, pro vice-chancellor for research and knowledge transfer at the University of Birmingham, told the Higher Education Policy Institute's annual research conference last week that it would be very difficult for panellists to avoid making relative judgements between submissions and to "drop" some submissions into the 2* ("internationally recognised") category.

This could give politicians and the public a false impression that the volume of high-quality research had declined since the 2008 research assessment exercise.

Bruce Brown, chair of main panel D, which covers the humanities, said this was a concern the panels shared.

But Ann Dowling, the chair of main panel B, which covers the physical sciences, pointed out that panel members' wide experience as peer reviewers meant they would bring a sense of how submissions fitted into the overall spectrum of research quality in their disciplines.

David Sweeney, director of research at the Higher Education Funding Council for England, which administers the REF, said he would "reinforce" the message that panels must make absolute rather than relative judgements.

But Professor Tickell warned that it would have to be repeated "early and often".

The REF panels are currently finalising their criteria and working methods following a consultation this autumn. Final versions will be released in January.

Professor Dowling said there was "unprecedented cooperation and communication" between the main panels, and that every effort was being made to meet respondents' calls for even more harmonisation of practices and terms used - not least to "prevent universities thinking there is something deeper" in terminological differences between panels.

She said respondents had also called for more clarity about the interplay between the two criteria - "reach" and "significance" - for assessing impact case studies.

She confirmed that doing very well on just one criterion would be sufficient for a high score.

She also clarified that it was not necessary for the person who did the underlying research to be submitted to the REF. She said the panels were thinking about how to assess the research underpinning the claimed impact, which is required to be at least 2* in quality.

"It would be great if institutions could come up with ways to convince the panels [that this threshold is met]," she added.

Professor Brown said the panels would try to be more precise about the "threshold criteria" for double-weighted submissions and the way multi-author papers would be treated.

He also insisted that panels' practice corresponded with official policy: "It is an urban myth that they don't stick to the guidance," he said.

paul.jump@tsleducation.com.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

Female professor

New data show proportion of professors who are women has declined at some institutions

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

John McEnroe arguing with umpire. Tennis

Robert MacIntosh and Kevin O’Gorman explain how to negotiate your annual performance and development review

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry