Hidden side to satisfaction

August 21, 2014

The contention that smaller institutions seem to do better than larger ones for student satisfaction scores in the National Student Survey (“Non-Russell Group institutions lead in satisfaction stakes”, News, 14 August) falls into the trap of what the statistician Howard Wainer calls “the most dangerous equation”. This is Abraham de Moivre’s equation, which describes how the averages of small samples vary more than those of large samples. When you are ordering averages from different sized groups, all other things being equal, you would expect to see smaller groups disproportionately represented at both the top and the bottom.

Wainer gives a nice example of how this misunderstanding can cost large amounts of money: having noticed that the top of school league tables was often dominated by smaller schools in the US, billions of dollars were spent on encouraging smaller schools, until it was noticed that smaller schools also dominated the bottom of the tables.

The same happens here: a simple plot of satisfaction score against number of responses in the most recent NSS results shows that smaller institutions dominate the bottom as well as the top. Indeed, the plot also suggests a small but positive relationship between size and satisfaction score.

There may be many very good reasons for preferring smaller institutions, but De Moivre’s equation isn’t one of them.

Adrian Simpson
Josephine Butler College, Durham University

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

PhD Position within Marine Structures

Norwegian University Of Science & Technology -ntnu

PhD Position in Biogas-Hydrogen

Norwegian University Of Science & Technology -ntnu

Director, Engineering and IT Learning Unit

The University Of Melbourne
See all jobs

Most Commented

Doctoral study can seem like a 24-7 endeavour, but don't ignore these other opportunities, advise Robert MacIntosh and Kevin O'Gorman

Laurel and Hardy sawing a plank of wood

Working with other academics can be tricky so follow some key rules, say Kevin O'Gorman and Robert MacIntosh

Matthew Brazier illustration (9 February 2017)

How do you defeat Nazis and liars? Focus on the people in earshot, says eminent Holocaust scholar Deborah Lipstadt

Improvement, performance, rankings, success

Phil Baty sets out why the World University Rankings are here to stay – and why that's a good thing

Warwick vice-chancellor Stuart Croft on why his university reluctantly joined the ‘flawed’ teaching excellence framework