TEF to limit use of ‘flawed’ NSS scores

Chair voices concerns over student satisfaction data as Cambridge’s students’ union joins NSS boycott

January 25, 2017
Judges hold up their score cards on the ice
Source: Getty
Just another number: ‘I do not think student satisfaction is an accurate proxy for teaching quality,’ said TEF assessment panel chair Chris Husbands

Student satisfaction scores will have only a limited impact on whether universities are judged gold, silver or bronze in the teaching excellence framework, the assessment panel’s chair has said.

As the University of Cambridge’s students’ union became the 25th union to boycott the National Student Survey (NSS) over the poll’s influence on the TEF’s ratings, Chris Husbands told an audience at the House of Commons that his team would “not be overweighting the NSS” when calculating which medal an individual university would receive.

Speaking at an event organised by the Higher Education Academy and the Higher Education Policy Institute on 24 January, the vice-chancellor of Sheffield Hallam University said that the TEF “should not place an overemphasis on satisfaction data”.

“I do not think student satisfaction is an accurate proxy for teaching quality,” said Professor Husbands, a former director of the UCL Institute of Education who was named chair of the TEF panel by the Department for Business, Energy and Industrial Strategy in August 2016.

Professor Husbands drew attention to US studies that show that student satisfaction scores are heavily influenced by non-academic factors, including the sex or ethnicity of a lecturer.

“Student satisfaction seems to be driven by the physical attractiveness of academics rather than anything else,” he said of the US research findings.

He added that the TEF panel would “not draw policy from a single data point” and that “all data” are “flawed” in some respect. However, he went on, the challenge was to recognise and understand the flaws, and to learn how the information could be used effectively.

Professor Husbands’ statement may allay the fears of many UK universities whose students have joined a National Union of Students-led movement to boycott the NSS. They hope to sabotage the TEF by rendering the NSS results unusable, thereby making it impossible for policymakers to differentiate tuition fees in future years.

Universities have already been made aware whether they would gain a gold, silver or bronze rating based on existing metrics, which include NSS scores, graduate employment data and student non-completion rates.

Universities must also submit a 15-page narrative statement on the institutional context of teaching. The statements, due by 26 January, will be used by the TEF panel to decide if an institution’s provisional rating based on metrics can be upgraded once its individual circumstances, its students and its local context are taken into account.

Ratings for UK universities will be announced in May 2017, with about 20 per cent of institutions likely to be awarded a bronze rating, 50 to 60 per cent silver and 20 to 30 per cent gold, according to a Department for Education report issued in September 2016.

At the HEA-Hepi event, one vice-chancellor, speaking under Chatham House rules, described the narrative statements as a “creative writing dialogue” with the Department for Education.

Another vice-chancellor wondered how the TEF could use NSS scores as its main measure of teaching excellence when they are widely recognised as poor proxies for teaching quality.

However, Professor Husbands defended the approach of using both metrics and written submissions, saying that the TEF was “underpinned by metrics” but also allowed for more nuanced analysis of teaching.

Evaluations would take into account teaching “based on institutional procedures and how they work to drive student outcomes”, he said.

jack.grove@tesglobal.com

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.

Related articles

Reader's comments (3)

US findings are presumably for module evaluations of single lecturers, not for whole-programme evaluations, so are of limited relevance. And the new NSS questions do include elements with good correlations to quality in t&L research, including the course being challenging and the quality of learning communities.
The TEF is an absolute disgrace. So some 70% of UK institutions will be debased by giving out bronze and silver awards this will hit overseas student numbers. It is a stupid bureaucratic exercise, based on flawed data and the people making the assessments are probably third rate bureaucrats. Nonsense.
The NSS is a poor measure of teaching quality, but a good measure of student quality. The most capable students almost always are the ones with the most positive outlook, while weak students tend to blame other factors. Of course the quality of one's peers is a factor in the student experience so the results are not entirely misleading. But a University can greatly improve its perceived quality simply by raising admissions standards, without changing teaching practice at all.

Sponsored