Small groups of aggrieved students will be prevented from ruining academic reputations under the Government's flagship student satisfaction survey, it was confirmed this week.
But by imposing a "reliability threshold" that sets a minimum response rate, the Higher Education Funding Council for England risks scuppering the entire exercise, it was claimed.
In its latest report on the planned survey of 280,000 final-year students, Hefce proposes that at least half the students in each subject area in each institution would have to respond for the results to be published.
Lee Harvey, who is widely accepted as the architect of student surveys, said that Hefce's proposal would mean that most results would have to remain secret.
Professor Harvey, director of the Centre for Research and Evaluation at Sheffield Hallam University, said that the pilot in 2004 produced a total response rate of only 40 per cent. He added that similar polls of final-year students regularly produced responses in the region of 30 per cent.
He explained: "If you set the response threshold at 50 per cent, your chances of getting the data you need is virtually zero. It is extremely difficult to get a 50 per cent response rate, extremely difficult."
He said he was "lucky to reach 30 per cent" in his own surveys of final-year undergraduates.
The survey was promised in the Government's White Paper on higher education of January 2003 for the autumn of that year. But it will not now appear until summer next year, having been blighted by concerns about the reliability of the results.
Concerns about low response rates have meant that the original plans to survey recent graduates, who may be hard to track down, have been scrapped in favour of polling final-year undergraduates before they have left university.
Professor Harvey complained that this would be less likely to encourage informed or honest responses, as undergraduates would not have completed courses and were more likely to want to ensure their courses come out looking good as they entered the jobs market.
Graeme Rosenberg, senior policy adviser for Hefce, said: "Yes, the pilot did get about 40 per cent overall but we only used telephone follow-ups at a few of the pilot higher education institutions.
"Where telephoning was used, the HEIs got nearer to 60 per cent. We are planning to use telephone follow-up at all HEIs."
He said the 50 per cent threshold was an initial suggestion that could change after further research.
THE QUESTIONNAIRE
Students will be asked to respond on a scale of one to five, with five indicating they "definitely agree".
They will be asked to respond to a total of 22 statements, which cover six aspects of a course: teaching quality; assessment and feedback; academic support; organisation and management; learning resources; and personal development. They include:
- Staff are good at explaining things
- Staff have made the subject interesting
- Assessment arrangements and marking have been fair
- I have received sufficient advice and support with my studies
- The course is well organised and is running smoothly
- The library resources and services are good enough for my needs
- The course has helped me to present myself with confidence.