Twenty-two universities have been selected to pilot the new system being designed to replace the research assessment exercise. But there are already concerns that the testing may be undermined by institutions manipulating the system to make themselves look good.
The research excellence framework will use a range of numerical measures, including the number of times academics' research is cited by their peers, in combination with peer review to judge the quality of a department's research.
The pilot institutions were chosen by the Higher Education Funding Council for England from 53 volunteers from across the UK. The testing of a range of models will start this summer, and the results will be available next spring.
The exercise is intended to help address many unanswered questions related to the design of the citations component - ranging from which subjects citations should be applied to and whether all an institution's staff and their papers should be included, to which citations database should be used.
Pilot institutions will provide Hefce with citations data for all staff eligible for inclusion in the 2008 RAE - regardless of whether the staff member was actually submitted - in 46 out of the 67 RAE subjects. Institutions can provide data for other subjects if they wish.
But worries are emerging that the institutions selected to be part of the pilot may treat the run-through as a game to boost their standing.
Trevor McMillan, pro vice-chancellor for research at the University of Lancaster - which did not get chosen - said he could understand that those selected might be a "little nervous" that they are seen to be in competition, but it was important for the pilot to be a genuine evaluation of citations.
"There is a chance that they (the selected institutions) could skew things by submitting only papers with a high number of citations or ignoring different types of output that may have fewer citations such as conference proceedings ... It has clearly got to be best for the sector as a whole if everyone plays a straight bat."
Bob Allison, the pro-vice chancellor for research at the University of Sussex, which is one of those selected for the pilot, said: "Our view - which we hope is the view of everyone in the pilot - is that this is about trying to help the funding council and the sector to come up with the right solutions for institutions as a whole rather than using it in some way to try to make a statement about how good your university is at research at this particular point in time."
The institutions chosen to be part of the pilot are the universities of Bangor, Bath, Birmingham, Bournemouth, Cambridge, Durham, East Anglia, Glasgow, Leeds, Nottingham, Plymouth, Portsmouth, Southampton, Stirling, Sussex, as well as Imperial College London, the Institute of Cancer Research, the London School of Hygiene and Tropical Medicine, Queen's University Belfast, The Robert Gordon University, the Royal Veterinary College and University College London.