Credibility fears delay students' staff audit

The government's flagship student satisfaction survey could be delayed for at least two years after the project steering group raised serious concerns that academics' and universities' reputations could be damaged by inaccurate and invalid results.

The government's higher education white paper in January said a comprehensive annual national survey of students' views on teaching quality would be published for the first time in autumn 2003. But a report by the project's steering group has warned that there must be piloting and further consultation to ensure the poll is valid.

The group has also rejected a suggestion - understood to have come from ministers eager to meet the white paper commitment - that a broad census of student satisfaction could be conducted and published during 2003. The committee feared that the census results would also be unreliable and, therefore, not publishable.

The report comes three months after The THES first revealed serious concerns in the group about potentially tiny response rates at the subject level, which could be, at best, meaningless and, at worst, unfairly damaging to the reputations of universities and their departments.

The report says: "We believe it will be vital to ensure that the questionnaire is sufficiently well developed if it is to produce results that will be valid and of genuine value to potential students. Therefore we strongly (recommend) that the proposed questionnaire should be tested in a pilot."

It also advises that "to avoid the real risk of low response rates preventing the production of reliable public information", an effective method of obtaining adequate response rates must be thoroughly tested.

Further, as the survey is likely to "incur substantial costs" to individual institutions and to central administrators, proper costing should be done before the project goes ahead.

The group said that the survey should publish findings at subject level in each institution in order to be meaningful. But only an unpublished two-phase pilot should take place this year, first to refine the questions themselves and then to test the response rates with a selected number of universities, it said.

Only a "developmental" survey could be undertaken in 2004, with published results, but "the level at which they could be disaggregated will depend on building up response rates", the report says.

"We recognise that surveys of this scale typically take a number of years to become fully established, and we envisage that the 2004 survey would also be developmental, leading to further refinements," it says.

A spokeswoman for the Department for Education and Skills conceded that the first survey would not be published this year.

She said: "We will proceed with our plans for the first-ever national survey of students' views. The information will be of critical importance in helping young people make better informed choices. It is essential that the survey gathers the right sort of information and that the methodology and findings are robust."