Rethink impact plans, think-tank tells Hefce

A number of the proposals for assessing impact in the research excellence framework need to be "rethought", an influential think-tank has argued.

In the first detailed critique since the proposals' release, the Higher Education Policy Institute adds that there is no "compelling" reason to halve the number of assessment panels. Hepi unveiled its analysis of the plans tabled by the Higher Education Funding Council for England at a Times Higher Education-sponsored conference this week on the future of research funding.

The REF, which will replace the research assessment exercise as the mechanism for allocating about £2 billion in annual quality-related research funding, resembles the RAE in that the assessment of academics' work will be based largely on peer review rather than metrics.

However, for the first time, "impact" will be included as a distinct element, counting for up to 25 per cent of assessment.

The proposals are currently in the consultation phase.

In its report, Proposals for the REF: A Critique, Hepi urges Hefce to tread carefully, asking whether the weighting is wise given the "serious methodological questions" facing the assessment of impact.

It says that, pending the results of a pilot, it seems "unnecessary and probably unwise" to place so much emphasis on an "untested and experimental process".

The report's author, Bahram Bekhradnia, director of Hepi and one of the architects of the RAE, warns that impact assessment will be a "serious additional burden" for universities.

Dr Bekhradnia is also critical of the suggested reduction in the number of subject panels from 67 to 30. He says it is "extraordinary" that Hefce is not consulting the sector on this change and calls on academics to give their views anyway.

"The proposal to reduce the number of panels would, on average, increase the amount of work that each remaining panel had to review," the report says.

"That is the reason for the proposal made in the (Hefce) consultation paper possibly to reduce from four to three the number of outputs submitted, but that would be at the expense of even less rigorous assessments."

The report goes on to argue that panels should determine impact weightings, rather than use a figure handed down from on high.

And it urges Hefce to widen the definition to include "academic impact", suggesting that the failure to do so would leave it open to charges of "philistinism".

Under the present proposals, it says, a historian could be valued more highly for researching and presenting a television series than producing a book that changes the way peers see their subject.

Graeme Rosenberg, Hefce's REF manager, said the plans were a work in progress, although a standard-impact weighting across disciplines was preferable for simplicity's sake.

"Decisions on the weightings will be taken after the consultation and pilot exercise concludes," he said.

He added that Hefce had not committed to a precise number of subject panels, although it did not want to keep the smallest ones.

zoe.corbyn@tsleducation.com