V-cs criticise 'unacceptable' inconsistency among reviewers

Universities can be publicly damned or praised depending on which reviewers turn up to undertake inspections, it was claimed this week.

Academics and vice-chancellors criticised the "inconsistency" in standards of different inspectors after the annual meeting with higher education watchdog officials in May.

A brief report on the meeting by the Quality Assurance Agency notes: "The agency should seek to get more consistency in the approaches of review teams."

But sources this week said that long-standing concerns that universities could be tainted for years on the personal whim of reviewers were coming to the surface.

"There is a strongly held belief in the sector that there is unacceptable variability in the calibre of auditors," said one. "Not so much with regards to their training, but with regards to their level of seniority and experience of higher education."

Under the QAA's peer-review system, universities are audited every six years. The audits are carried out by a team of between four and seven auditors who are selected by the QAA, based usually on nominations from universities.

Geoffrey Alderman, a former QAA reviewer and head of quality assurance at the American InterContinental University, said: "In my experience, you can have auditors who are minded to bend over backwards to excuse or play down matters because the institution they are visiting is prestigious. Matters glossed over in an old university would be condemned if they occurred at a new university."

He said auditors sometimes had "hobby horses" they wanted to make an issue of "come hell or high water" or had in-built suspicions of a university's mission.

Kel Fidler, vice-chancellor of Northumbria University, said: "My experience across all sorts of peer assessment is that there is sometimes variation in the approach of members of teams, and potentially, therefore, teams themselves.

"Examples are of members having a bee in their bonnet about a particular area, which then dominates matters; of members not appreciating the scope of an assessment - talking to a random selection of students outside the student union while assessing a particular subject area in which said students were not involved, and so on."

He said it was crucial to provide good training for all assessors and to ensure that teams had a strong, experienced chair.

The QAA said in a statement about the meeting this week: "A handful of people mentioned that, in their view, the personal styles of individuals conducting reviews might influence the approach taken by the review team.

This was not offered as a criticism. Some felt that it was an advantage for review teams to be able to vary their approaches to suit the styles and diversity of institutions, while others said they would prefer a more standardised approach.

"There was no suggestion from our subscribers that differences in style had had any effect on the judgements of the review teams."