Critics fuming over RAE criteria

Academics say that interdisciplinary research, early-career staff and the UK's reputation will suffer in RAE 2008. Anthea Lipsett reports

Academics have voiced concerns that the switch to four new research grades in the 2008 research assessment exercise could damage Britain's international research reputation.

As the consultation on criteria for the 2008 exercise closed, Peter Cotgreave, director of the Campaign for Science and Engineering in the UK, said that the boundaries between the top international research ratings would be more or less arbitrary and would damage the UK's international standing.

"We have spent 15 years explaining to people around the world what it means and that 5* is what really matters. This time round, no one will get one.

Our international reputation will suffer. People are furious," he said.

Stuart Palmer, pro vice-chancellor of research at Warwick University, agreed. "It could look like there's a sudden drop in research standards in the UK," he said.

Concerns also emerged about the reading load involved, interdisciplinary and applied research and the treatment of new entrants to the exercise.

The RAE panels start their third round of meetings in October to finalise the criteria for judging research. Research heads want them to clarify the definitions of the four new grades of 4*, for top research, 3*, 2* and 1*.

The top three grades reflect research of an international standard.

Dame Nancy Rothwell, pro vice-chancellor for research at Manchester University, said: "Going from a single grade to a profile was welcome, but distinguishing between three categories of 'international' seems impossible."

She said the disparity in the weightings between panels was a concern and noted that the emphasis placed by most panels on research outputs rather than people will mean a huge reading load for panellists. "It will be impossible for some panels to read them all," Dame Nancy said.

Mike Cruise, Birmingham University's pro vice-chancellor for research, said: "People are grappling with what 'world-beating' means. Taken to the extreme, there can be only one world leader in any given subject. The choice of wording will have to be defined."

Graham Dockray, Liverpool University's pro vice-chancellor for research, said: "Clearly some panels have thought very hard and deeply and have produced very good descriptions for each of the four grades. Others haven't attempted to and that's an issue."

He said panels could have been more explicit on the amount of research young entrants at different stages in different disciplines would be expected to produce.

Research heads want more clarification on how new entrants will be judged and on interdisciplinary and applied research.

Karen Ness, Glasgow University's director of research strategy, said:

"There's a lack of consistency and clarity about how young entrants will be handled."

But Lawrence Freedman, pro vice-chancellor of research at King's College London, said panels would be sufficiently alive to the issue of how to deal with new entrants to "handle it reasonably sensitively".

Interdisciplinary research has proved problematic in the RAE. Many suspect sub-panels will look to advisers rather than cross-refer research because of the differences in emphasis placed on research outputs, environment and esteem.

Despite efforts to address inter-disciplinary and applied research, academics are still concerned about how it will fare. Professor Cruise said: "The world will be watching how well interdisciplinary research does because it's exciting research. There's a fear that it will not be totally successful."

Professor Freedman said: "An awful lot of the most interesting work takes place across disciplines. A real test will be if these people are given the credit for it. It will be much easier to see this time around what work is well regarded and what is considered to be a bit marginal."

anthea.lipsett@thes.co.uk

PANELS LOOK BACK TO 2001

Some research assessment exercise panels plan to use data from the 2001 exercise to test their criteria, it has emerged.

Keith van Rijsbergen, chair of the computer science panel, said his panel would meet to "normalise" guidelines. "There are lots of judgments to be made, and because the RAE rules have changed, there's some necessity to spend time judging how individual assessments will be made," he said.

His panel is trying to refine the process using data from the 2001 RAE to help resolve disagreements over judgments. "We are simply trying to go through the process of using the new rules and then it's not a question of what did you get but how did you get there," he said.