Exam questions that fail to challenge students as well as questions that are repeated year after year are among the poor assessment practices that continue to threaten standards on university courses in England, quality watchdogs have warned.
After the world's largest academic quality inspection exercise, the Quality Assurance Agency has concluded that some universities have failed to deal with repeated warnings about assessment weaknesses.
Problems that persisted throughout the nine-year teaching quality assessment (TQA) exercise, which ended in 2001, included: "insufficiently challenging" exam questions; repetition of questions year after year; an over-reliance on traditional exams; a failure to double-mark exams; and "perfunctory" and often late feedback for students.
The overview report, Learning from Subject Review , published on Friday, rounds up the findings of 2,904 visits to university and college departments by more than 5,000 inspectors between 1993 and 2001, which covered 62 subject areas.
"In general, reports emanating from the cycle attest to the positive reputation of higher education provision in England and Northern Ireland, and that the vast majority of students receive education of a good quality," the overview report says.
"However, the reports also point to a number of areas where concerns about the quality of provision remain. Assessment is one of the aspects of higher education provision in which there remains scope for improvement," the report says.
The TQA, renamed subject review during the exercise, examined six aspects of departmental provision: curriculum design and content; teaching, learning and assessment; student progression and achievement; student support and guidance; learning resources; and quality assurance management.
Each of these six aspects was given a grade from one, indicating failure, to four, indicating high quality.
Throughout the exercise, the lowest marks were persistently given to the teaching, learning and assessment aspect.
"The issues identified by the reviewers related, in the main, to deficiencies in assessment practices, rather than to weaknesses in teaching and learning. In general, the reviewers found the quality of assessment practices to be limited in many respects," the report says.
It adds that reviewers repeatedly highlighted "the narrow range of assessment methods employed", suggesting that an over-reliance on traditional exams "did not allow students to demonstrate their achievement of all learning outcomes".
This problem was particularly marked in the early stages of the exercise but "in recent review visits, the reviewers continued to point to the narrow range of assessment methods employed".
There were also repeated concerns that "assessment tasks did not always constitute an adequate intellectual challenge".
The reviewers said some departments failed to "distinguish appropriately between the intellectual demands required at different levels of study".
Others used exam questions "seeking only the recall of information and knowledge, rather than higher-level cognitive skills".
There were also problems with marking. In a "significant minority" of cases in the later part of the exercise, "the marks awarded were not consistent with the marking criteria".
One of the most consistently weak areas of assessment practice was the feedback given to students. "In every subject overview report for the two final rounds of visits, the reviewers identified deficiencies in the feedback given to students, [this] included the perfunctory nature of feedback, the absence of constructive comment, particularly for weaker students, and the failure to make timely comment so that students could improve their subsequent performance."
The report concludes: "Despite the evidence of developmentsin assessment strategies and practices, the reviewers, even in the later round of visits, saw this as the area most in need of further consideration by institutions."
Peter Williams, chief executive of the QAA, declined to comment.
Was it all worth it?
It cost an estimated £250 million a year and lasted for the best part of a decade. Under the teaching quality assessment exercise, 5,700 trained quality inspectors made almost 3,000 separate visits to university departments, covering 62 subjects.
It was, according to the Quality Assurance Agency: "The largest and most comprehensive exercise of its kind ever undertaken" - designed to ensure accountability for the public funds invested in universities.
Of the 2,904 review visits, the inspectors found 16 cases (0.5 per cent) where departments were not meeting their own aims and objectives. Of these, one department was judged after a re-review to be permanently failing.
The TQA was to "provide a link between funding and the enhancement of quality". But as today's QAA report says: "Neither the Higher Education Funding Council for England nor the then Department for Education and Learning and its predecessors in Northern Ireland ever used subject review outcomes to direct extra funding specifically to high-scoring providers."
Nor did they ever withdraw funds from failing providers.
Geoffrey Alderman, a former TQA reviewer, was responsible for preparing his university for its frequent, traumatic inspections as head of quality assurance at Middlesex Univer-sity.
"It would be foolish to say that no good came of the exercise for at least some departments, but the taxpayer did not get value for money," he said.
"It was an enormous waste of expenditure."
Professor Alderman also believes that the results of the exercise were unreliable. "An entire industry developed and I made a bit of money training people how to play the game to gain the maximum points," he said.
But the TQA was not a waste of time and money, the QAA says. The exercise provided invaluable public information on the quality of university departments, it helped spread best practice and, perhaps most important, it provided the "extensive evidence for the high quality of higher education", creating the "sound base" for the new, light-touch era of quality assurance.
It also - unintentionally - formed the basis for various newspaper university league tables.