Philosophy scores add to QAA criticism

A quality assurance expert this week suggested that a "cartel" of quality inspectors was artificially bolstering the performance of philosophy departments in the Quality Assurance Agency's teaching quality assessment exercise. An analysis by The THES showed that the average assessment score in the field is 23.7 out of 24, with excellent ratings in all departments so far.

An emerging gulf between the performances of further education colleges new to the system and the more experienced universities has raised separate allegations that gamesmanship is rendering the exercise meaningless.

This week, Roger Cook, academic development adviser at Napier University, said the latest statistical analysis "further calls into question the validity of the whole exercise". The TQA exercise, known formally as subject review, will come to an end in December after almost a decade. The regime has already faced serious criticism for costing institutions up to £300 million a year in staff time and red tape.

The THES analysis of subject review results for philosophy departments shows way above average performance. Of 13 reports so far released on individual departments, ten have been given the maximum 24 out of 24. Of three departments that failed to win top marks, two received 23 and the other received 22. This gives an average performance among philosophy departments of 23.7 out of 24, compared with an overall average for the entire exercise of about 20.7.

Philosophy has emerged as the most extreme case of the alleged "cartel" abuse, in which the QAA's peer-reviewers in small subject areas are said to be over-generous.

"The extent to which this subject is seriously out of line with anything else, and the fact that it is a small area known to be under threat in various universities, raises concerns of a cartel," Mr Cook said.

Geoffrey Alderman, a former QAA inspector, said: "I suspect the philosophers are poking fun at the entire methodology."

Similar allegations have been made following a THES analysis of the review of business and management studies departments, which shows a number of anomalies.

Of 17 reports published on the QAA's website for the 1999-2001 round, university departments have scored an average of 22.2 out of 24. This compared with an average score for further education colleges of 19.25 out of 24.

Mr Cook said that this situation could support concerns that university departments, which have been subjected to reviews for almost a decade, have learnt to "play the game" and present themselves in the way QAA reviewers prefer.

This contrasts with colleges, which have not yet learnt the tricks of the trade, having only recently been subjected to QAA scrutiny following a change in the way sub-degree provision is funded.

"It shows the university-college divide is alive and well," Mr Cook said. He also said that the statistics suggested that institutions' resources, which are supposed to be factored out of judgements, were still dominating subject review scores.

The QAA was not willing to comment.