REF panellists must be absolutely clear: quality in the UK is not in decline

Research excellence framework panels must be sure to make absolute rather than relative judgements of submissions to avoid the impression that the quality of the UK research base is in decline, a pro vice-chancellor has warned.

December 1, 2011

Universities' expectation that future funding formulas will take into account only work rated 3* ("internationally excellent") and 4* ("world-leading") means that many are planning to submit only those outputs they believe fall into these categories to the 2014 REF.

But Adam Tickell, pro vice-chancellor for research and knowledge transfer at the University of Birmingham, told the Higher Education Policy Institute's annual research conference last week that it would be very difficult for panellists to avoid making relative judgements between submissions and to "drop" some submissions into the 2* ("internationally recognised") category.

This could give politicians and the public a false impression that the volume of high-quality research had declined since the 2008 research assessment exercise.

Bruce Brown, chair of main panel D, which covers the humanities, said this was a concern the panels shared.

But Ann Dowling, the chair of main panel B, which covers the physical sciences, pointed out that panel members' wide experience as peer reviewers meant they would bring a sense of how submissions fitted into the overall spectrum of research quality in their disciplines.

David Sweeney, director of research at the Higher Education Funding Council for England, which administers the REF, said he would "reinforce" the message that panels must make absolute rather than relative judgements.

But Professor Tickell warned that it would have to be repeated "early and often".

The REF panels are currently finalising their criteria and working methods following a consultation this autumn. Final versions will be released in January.

Professor Dowling said there was "unprecedented cooperation and communication" between the main panels, and that every effort was being made to meet respondents' calls for even more harmonisation of practices and terms used - not least to "prevent universities thinking there is something deeper" in terminological differences between panels.

She said respondents had also called for more clarity about the interplay between the two criteria - "reach" and "significance" - for assessing impact case studies.

She confirmed that doing very well on just one criterion would be sufficient for a high score.

She also clarified that it was not necessary for the person who did the underlying research to be submitted to the REF. She said the panels were thinking about how to assess the research underpinning the claimed impact, which is required to be at least 2* in quality.

"It would be great if institutions could come up with ways to convince the panels [that this threshold is met]," she added.

Professor Brown said the panels would try to be more precise about the "threshold criteria" for double-weighted submissions and the way multi-author papers would be treated.

He also insisted that panels' practice corresponded with official policy: "It is an urban myth that they don't stick to the guidance," he said.

paul.jump@tsleducation.com.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Daniel Mitchell illustration (29 June 2017)

Academics who think they can do the work of professional staff better than professional staff themselves are not showing the kind of respect they expect from others

celebrate, cheer, tef results

Emilie Murphy calls on those who challenged the teaching excellence framework methodology in the past to stop sharing their university ratings with pride

Senior academics at Teesside University put at risk of redundancy as summer break gets under way

Tef, results, gold, silver, bronze, teaching excellence framework

The results of the 2017 teaching excellence framework in full. Find out which universities were awarded gold, silver or bronze

Thorns and butterflies

Conditions that undermine the notion of scholarly vocation – relentless work, ubiquitous bureaucracy – can cause academics acute distress and spur them to quit, says Ruth Barcan