The magnitude of the task facing the Higher Education Funding Council for England if it is to assess the impact of academics' work robustly in the research excellence framework has been revealed.
A report commissioned by Hefce from the consultancy RAND Europe examines international approaches to the subject. Capturing Research Impacts concludes that a proposed Australian model, which Labor scrapped when it took power in 2007, provides "a promising basis" for the REF.
The Research Quality Framework proposed measuring impact on a three-point scale based on peer review of the portfolio of work from "research groupings" of five or more people. The report says the RQF "was abandoned for political not feasibility reasons ... (and) provides the 'best fit' against the emerging criteria for REF".
But while the Australian model is praised for being credible and acceptable to academics, encompassing a wide range of impacts and being applicable across all disciplines, the report warns that it could place a major burden on institutions and create "undesirable perceptions and incentives".
The report goes on to set out the challenges facing Hefce, which is urged to be clear about its "strategic intent" in assessing impact and also to spell out the effect of more concentrated research funding.
It says Hefce needs to define impact and timescales clearly; have a firm idea of the units it wants to assess; acknowledge the challenge of attributing impact; assess the administrative burdens involved; and clarify the role of research users.
It also recommends that REF panels be given "some flexibility" in quality, impact and environment weightings. Hefce has suggested that impact should count for 25 per cent in the REF, which will replace the research assessment exercise in distributing £1.5 billion in research cash in England every year.
A Hefce spokesman said a pilot exercise under way at 29 universities was taking account of the issues raised in the report and the responses to the REF consultation, which closed last month.