The views of ten students could be enough to make or break the reputations of university departments and the academics who work in them, according to a consultants' report on the planned annual student satisfaction survey, writes Phil Baty.
Proposed survey questions, seen by The THES, would ask final-year students annually from next year to rate their courses and lecturers' abilities in a number of areas on a scale from 1 to 5.
The planned 28-question survey has been published as part of a report for the Higher Education Funding Council for England by consultants SQW and pollsters NOP.
The report admits to serious concerns about poor response rates making the survey invalid. It concludes: "Our view is that response rates as small as ten, to a census survey, could still convey meaningful information." In very early pilots, only 20 out of 210 graduates - 10 per cent - responded to questionnaires.
Lecturers' leaders expressed alarm this week that entire departments could be publicly damned or praised on the say-so of just a handful of students.
Tom Wilson, head of the universities department at lecturers' union Natfhe, said: "It's really disgraceful that a bunch of ten students getting together over a pint can make a judgement that could destroy the reputation of an entire department, or even a university, and seriously damage recruitment."
The report acknowledges the survey's unpopularity with academics. It says:
"There is widespread concern that the national survey would lead to the creation of another league table, and one that might have a significant influence on external perceptions of the institution."
The report says the national survey should publish judgements on teaching quality down to the level of 19 individual subject areas in each university.
Students will be asked to indicate their agreement - on a five-point scale from strongly disagree to strongly agree - with a series of statements relating to six key areas of their learning experience: good teaching; generic skills; clear goals and standards; appropriate workload; support and advice; and learning resources.
They will be asked to rate lecturers' ability to motivate students, to explain things and to make the subject interesting. They will also be asked to judge how well the course developed their skills and whether it delivered what was expected.
An average score is likely to be given for each of the six areas. Finally, students will be asked to indicate, on the same 1 to 5 scale, their agreement with the statement: "Overall, I was satisfied with the quality of my course."
The report recommends that a questionnaire be posted each year to all 350,000 graduating students before they sit their final exams. But it warns that even with this universal approach, those responding could have opposite views to those who fail to respond.
The survey could also be blighted by a tendency of students at high-ranking universities who have higher entry qualifications to judge their departments more harshly than students at less prestigious universities who have lower expectations.
The report acknowledges that the survey, which consultants estimate will cost at least £634,000 a year, will also fail to provide any quality enhancement opportunities.
A spokesman for Hefce said: "We are piloting the survey this summer and one of the aims is to test how far results can be disaggregated and still remain reliable. The steering group for the project expressed concern about this, and we accepted their conclusions."