Rethink impact plans, think-tank tells Hefce

Hepi criticises the REF's 'untested' procedure and plans to cut subject panels. Zoë Corbyn writes

October 15, 2009

A number of the proposals for assessing impact in the research excellence framework need to be "rethought", an influential think-tank has argued.

In the first detailed critique since the proposals' release, the Higher Education Policy Institute adds that there is no "compelling" reason to halve the number of assessment panels. Hepi unveiled its analysis of the plans tabled by the Higher Education Funding Council for England at a Times Higher Education-sponsored conference this week on the future of research funding.

The REF, which will replace the research assessment exercise as the mechanism for allocating about £2 billion in annual quality-related research funding, resembles the RAE in that the assessment of academics' work will be based largely on peer review rather than metrics.

However, for the first time, "impact" will be included as a distinct element, counting for up to 25 per cent of assessment.

The proposals are currently in the consultation phase.

In its report, Proposals for the REF: A Critique, Hepi urges Hefce to tread carefully, asking whether the weighting is wise given the "serious methodological questions" facing the assessment of impact.

It says that, pending the results of a pilot, it seems "unnecessary and probably unwise" to place so much emphasis on an "untested and experimental process".

The report's author, Bahram Bekhradnia, director of Hepi and one of the architects of the RAE, warns that impact assessment will be a "serious additional burden" for universities.

Dr Bekhradnia is also critical of the suggested reduction in the number of subject panels from 67 to 30. He says it is "extraordinary" that Hefce is not consulting the sector on this change and calls on academics to give their views anyway.

"The proposal to reduce the number of panels would, on average, increase the amount of work that each remaining panel had to review," the report says.

"That is the reason for the proposal made in the (Hefce) consultation paper possibly to reduce from four to three the number of outputs submitted, but that would be at the expense of even less rigorous assessments."

The report goes on to argue that panels should determine impact weightings, rather than use a figure handed down from on high.

And it urges Hefce to widen the definition to include "academic impact", suggesting that the failure to do so would leave it open to charges of "philistinism".

Under the present proposals, it says, a historian could be valued more highly for researching and presenting a television series than producing a book that changes the way peers see their subject.

Graeme Rosenberg, Hefce's REF manager, said the plans were a work in progress, although a standard-impact weighting across disciplines was preferable for simplicity's sake.

"Decisions on the weightings will be taken after the consultation and pilot exercise concludes," he said.

He added that Hefce had not committed to a precise number of subject panels, although it did not want to keep the smallest ones.

zoe.corbyn@tsleducation.com

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Most Commented

James Fryer illustration (27 July 2017)

It is not Luddism to be cautious about destroying an academic publishing industry that has served us well, says Marilyn Deegan

Jeffrey Beall, associate professor and librarian at the University of Colorado Denver

Creator of controversial predatory journals blacklist says some peers are failing to warn of dangers of disreputable publishers

Hand squeezing stress ball
Working 55 hours per week, the loss of research periods, slashed pensions, increased bureaucracy, tiny budgets and declining standards have finally forced Michael Edwards out
Kayaker and jet skiiers

Nazima Kadir’s social circle reveals a range of alternative careers for would-be scholars, and often with better rewards than academia

hole in ground

‘Drastic action’ required to fix multibillion-pound shortfall in Universities Superannuation Scheme, expert warns