Referees deliver online scolding

The Government's student website is beset by problems in its three key areas

The external examiner for Liverpool John Moores University clearly had something he wanted to get off his chest.

"There are too many failed dissertations this year," the anonymous referee writes in his official report on the university's social work courses. The report, published on the new Teaching Quality Information website, required the examiner to give simple yes/no answers to basic questions about the standards of courses and the fairness of assessment procedures. But he opted to provide rather more detail.

It was unclear, the referee says, whether the high failure rate was "due to standards set", or to poor "teaching input", or even to "the course team being preoccupied by undertaking the necessary preparations for the new three-year degree course".

But, most damning of all, he suggests it could have been due to "the lack of effective overall leadership for social work at university senior management level".

"This programme used to be at the forefront of social work practice standards," he adds. "It is my contention that standards have not been effectively sustained since the departure of Michael Preston-Shoot" - the former professor of social work and social care who departed LJMU two years ago for Luton University.

"The issue is overall effective leadership, which is particularly pertinent as social work at a national level is at a critical stage of development."

As if this were not enough, the examiner adds one final insult to the university's injury with a comment that could prompt future student complaints and even legal action against the university. A cock-up with the university's information technology system, he claims, meant that decisions about students' results were made "on the basis of incomplete information.

This is unacceptable as it leaves the programme/university open to issues of contestability."

This is just one of the warts-and-all accounts by academic external examiners that have been released for public consumption for the first time on the TQI website. On the face of it, publishing such reports to inform prospective students would seem a laudable thing to do. The result, however, will send shock waves across the academic sector.

University heads heaved a collective sigh of relief when the funding councils agreed in October 2003 to pull back from their original demand that the full, unadulterated and previously secret examiners' reports all be revealed.

Funding chiefs heeded the argument that the full publication of externals'

reports would destroy a delicate system that relies on trust, goodwill and above all, candid and frank confidential advice. Instead, only sanitised summaries of reports would be published, with little more than a series of tick-boxes giving basic guarantees about the quality of courses.

That relief now looks badly misplaced. The Times Higher's analysis of the (albeit incomplete) material on the TQI site shows that while some examiners have complied with the tick-box approach, others have taken an opportunity to vent some spleen. A study of the reports at just one university, LJMU, reveals numerous problems across several subject areas.

The examiner for LJMU's healthcare and specialist practitioner BA degree says: "There is a general concern that students who struggled with English have difficulty in expressing themselves in an understandable way. This can detract from their overall performance. There are still many students who have difficulty in relating theory to practice..."

The examiner for the university's maths, statistics and computing BSc, meanwhile, says final-year projects were "not very good".

"There were many errors in general grammar and construction of technical reports, including poor punctuation and the use of the first person," he said. "I would suggest grammar guidelines be handed out to all students. " And the examiner for the sports technology BSc warns that some examiners "recycle questions on a three-yearly basis. I believe this reflects lack of subject development, which is disappointing."

Finally, on the university's BA hons in international business studies, the examiner said that the process of assessment and examination was not sound or fair. "I felt that higher marks have been easily awarded thus I have suggested lowering several marks. At times, the marks seem to be influenced by tutors' expectations rather than the actual quality of students'

performance or work." He says that the marks were amended and that he is "now happy with all marks".

A spokeswoman for LJMU said: "As part of our quality audit process, we have analysed all the external examiners' reports for 2003-04. In 99 per cent of cases, external examiners agree that the standards set by the university for specific awards are appropriate for the subject and level.

"In 98 per cent of cases, the standards of student performance were comparable to other UK institutions and in 99 per cent of cases the processes for assessment, examination and the determination of awards was sound and fairly conducted.

"These results confirm that the standards and quality of teaching and assessment at LJMU compare favourably to those of other UK HE institutions," she added.

"LJMU is committed to continually raising the standards of our teaching and learning services. If the quality issues raised by external examiners relate to possible failings in operational procedures, the university is committed to taking appropriate action.

"LJMU is in the process of writing to all of our external examiners, and particularly those who raised concerns, to inform them of the range of actions we are taking to ensure that the university continues to meet sector benchmarks for teaching and assessment."

phil.baty@thes.co.uk

Details: www.tqi.ac.uk