TEF data: so what can we count on?

Scrutiny of datasets informing the teaching excellence framework will ramp up, and rightly so, given the high stakes and risk of misuse

May 26, 2016
Data being analysed under magnifying glass (illustration)
Source: iStock

We live in the age of data. A word that once evoked the deadliest of dead-end jobs (I once spent a month doing data entry in an NHS deanery) now speaks of data scientists who rule the world from trendy cafes, powered by skinny soya lattes.

As the American statistician and management guru W. Edwards Deming put it: “In God we trust, all others bring data”. This is a mantra that’s revolutionising business, dismantling old industries and making billionaires of digital pioneers. It is also, inevitably, finding its way into public policy, and changing the fabric of academia, one of the most data-hungry professions around.

But data must be handled carefully, must be clean and meaningful, and understood and used in the right way – lessons we’re very clear about as the publishers of the Times Higher Education World University Rankings. And when it directly affects the prosperity of individuals or institutions, the integrity of data has to be protected as never before.

This is front of mind in the UK because of the forthcoming teaching excellence framework, which we now know will be tied to university funding (by informing tuition fees) despite intensive lobbying by universities and students to resist such a link.

Last week’s White Paper softened the approach, giving a two year bedding-in period, but we should be in no doubt that more aggressive financial incentives are coming.

One result is that the scrutiny of the data fuelling the TEF will also ramp up, and in our news pages this week we highlight just one of the potential chinks.

Our investigation reveals that the funding council has looked into claims that institutional staff have tried to get undergraduates to inflate their course ratings in the National Student Survey (one of the TEF metrics) on 30 occasions over the past seven years.

This isn’t a huge number in the scheme of things (and only four were serious enough to warrant NSS results being withheld from data sources such as Unistats), but it shows that there’s already pressure to a degree that is warping individuals’ actions.

That pressure will intensify dramatically with the introduction of the TEF.

The reality is that there will always be flaws with datasets of this kind, and the risk of misuse and abuse is equally inevitable.

Most academics will also accept that imperfect data can be useful if handled in the right way (although it has to be said that government has a pretty ropey record in this regard, particularly when it comes to supposedly data-led reform of the NHS).

But while accountability and competition are indeed vital components in universities’ success, there is an underlying worry here: that a toxic culture develops in which everything is measured and the measurement becomes an end in itself.

Deming’s quote about the divine nature of data is counterbalanced by his “seven deadly sins” of management.

Among them are “lack of constancy of purpose”, “evaluation by performance, merit rating, or annual review” and “management by use only of visible figures, with little or no consideration of figures that are unknown or unknowable”.

These are equally important lessons as higher education is re-envisioned.


You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

PhD Position in Archaeology and Cultural History

Norwegian University Of Science & Technology -ntnu

PhD position in Energy and Process Engineering

Norwegian University Of Science & Technology -ntnu

PhD position in Energy and Process Engineering

Norwegian University Of Science & Technology -ntnu

PhD position in Industrial Energy Efficiency

Norwegian University Of Science & Technology -ntnu

Postdoc in Traffic Engineering

Norwegian University Of Science & Technology -ntnu
See all jobs

Most Commented

Doctoral study can seem like a 24-7 endeavour, but don't ignore these other opportunities, advise Robert MacIntosh and Kevin O'Gorman

Matthew Brazier illustration (9 February 2017)

How do you defeat Nazis and liars? Focus on the people in earshot, says eminent Holocaust scholar Deborah Lipstadt

Improvement, performance, rankings, success

Phil Baty sets out why the World University Rankings are here to stay – and why that's a good thing

Warwick vice-chancellor Stuart Croft on why his university reluctantly joined the ‘flawed’ teaching excellence framework

people dressed in game of thrones costume

Old Germanic languages are back in vogue, but what value are they to a modern-day graduate? Alice Durrans writes