Universities could face "huge" costs managing the system that will replace the research assessment exercise, an expert has warned.
The research excellence framework was designed to help "reduce the administrative burden on universities" by replacing the RAE's peer review with a system that makes greater use of statistical indicators of research quality, such as data on the number of times an academic's published research is cited.
The Higher Education Funding Council for England is moving into a pilot stage for the REF, during which it will decide which data are to be collected, which databases will be used and how much information individual institutions will have to provide, analyse and verify.
But in a conference speech last week, Linda Butler, head of the Research Evaluation and Policy Project at the Australian National University, said that when a similar metrics system was implemented in Australia in the 1990s, universities bore most of the data-collection burden.
"(The particular methodology chosen) is going to have direct implications on how much it is going to cost each institution and Hefce itself - not just in terms of purchasing data ... but (also) the implementation costs and staff time," Ms Butler told the conference, "Beyond the RAE 2008: Bibliometrics, League Tables and the REF".
Another conference participant said that even if Hefce ultimately takes responsibility for data-gathering in the REF, individual universities will still undertake parallel exercises to ensure that they are not missing out on funding.
Ms Butler said that the development of institutional information systems is complex and takes time, and the "choice between simplicity and complexity has huge implications for the costs of the exercise".
Ms Butler warned that vice-chancellors lobbying for particular research evaluation systems based on what they believed would benefit their universities often had "no understanding" of the cost involved.
A pre-conference survey of delegates showed that their biggest worry about the new metrics system was "getting accurate and verifiable data" - 28 per cent described this as their main concern. About 20 per cent said their main fear centred on the "time and workload" implications.
Ms Butler's concerns were reiterated by Jonathan Adams, director of the data analysis firm Evidence Ltd, who said: "The cost will almost certainly be larger than we are expecting." He also expressed doubts about how long metrics could accurately reflect research excellence: "As soon as you start to use an indicator for public policy, it will lose the information value that made it a good metric in the first place."
Graeme Rosenberg, Hefce's REF pilot manager, told the conference that the sector accepted that a way of making a metrics system work could be found.