'Vocal minority' could ruin faculty reputations

The views of 20 students could soon be enough to make or break the reputation of university departments and the academics who work in them under plans to reform the landmark National Student Survey.

Until now, survey results were not published unless at least 50 per cent of students on a course, or a minimum of 30 students, had responded.

But under heavily criticised reforms put out to consultation this week, the so-called "reliability threshold" would be reduced to 40 per cent, or 20 students, to allow more results to be published in greater detail.

The recommendation was sent out this week by the Higher Education Funding Council for England, following a report published this week by the Quality Assurance Framework review group.

The consultation paper recognises that the original 30-student threshold was "introduced to ensure that the results were reliable" and was based on "the expert judgment" of the NSS steering group.

But it says that modelling of the results of the first two surveys has shown that the threshold could be lowered without "significantly affecting the reliability of the results". The change would allow the next survey to be published at a greater level of subject detail than currently, the paper argues.

Harvey Goldstein, a professor of statistics at Bristol University who was part of the original steering group, said that any lowering of the reliability threshold "obviously reduced" the reliability of the survey, and left departments "vulnerable" to being judged based on the views of a vocal minority of students.

Lee Harvey, director of the Centre for Research and Evaluation at Sheffield Hallam University, said there would be "greater distortions".

"Imagine, if you wanted to get a good response, just persuade 20 students to give you a five rating," he said. "The scope for manipulation of results has just been increased significantly. In short, the NSS is a spurious waste of taxpayers' money with no real benefit."

As well as planned reforms to the student survey - which will be expanded to cover Department of Health-funded students and will continue to be carried out every year - the review group, led by Dame Sandra Burslem, agreed a complete overhaul of the Teaching Quality Information website, which was launched last year to help students choose where to study.

A redesigned TQI website, which will be run by the Universities and Colleges Admissions Service, will be launched in the summer of 2007.