NSS and DLHE reform: don’t throw the baby out with the bathwater

John Cater argues that higher education would be poorer for the loss of student survey data

March 7, 2016
Survey responses

The phones have been ringing for the past month and, across the country, four-fifths of last summer’s graduates have answered the Destinations of Leavers from Higher Education survey, as they have for the past decade. And, across the campus, almost three-quarters of this year’s finalists are answering at least 23 questions about their academic experience in the National Student Survey. A dozen years, almost two dozen questions, five choices and more than 250,000 respondents every spring. 

So we now know what the vast majority of graduates are doing six months into their post-graduation lives, and we will soon know what the cohort behind them think of their university experience. Yet, we decry, both studies and are in grave danger of dumping a decade or more of time-series data. Why?

As academics we’re critics, bred to focus on limitations and shortcomings. An evening with us is like a date with an actuary. But where else will you find such a welter of information provided by those with real experiential insight? Of course, the DLHE isn’t perfect; not everyone’s long-term career is going to be determined by what they are doing in the first winter of discontent. But it gives us a vital indicator of the diminishing number affording postgraduate study, of the initial earnings of this April’s taxpayers, and of the first steps in the labour market. Yes, it should be supplemented by time-series data that follows the individual, not just the sector, but this is not an argument for its disappearance.  

The future of the NSS has also been subject to review; it will be different and it may well also disappear, at least in its present form. We already know that question 23 on students’ unions – a vital measure of student engagement – has been consigned to the dustbin of history. The Higher Education Funding Council for England is warning institutions and departments against exerting undue influence on this year’s NSS respondents, having identified abuses in the past. But reported instances have been rare and are an argument for a different approach to data collection focused on the individual, not the institution – not data dismissal.

Basic questionnaire techniques, such as asking some questions in the negative, would help to eliminate the column-tickers, those who return 22 unthinking or uncritical “fives” or, more rarely, 22 unthinking or malevolent “ones”. Or, as an alternative, eliminate the outliers from the calculation of institutional means so beloved of some league table compilers. And persuade other compilers to stop using two categories (scores of four and five) from one question (overall satisfaction) and representing this as a holistic analysis of anything more than one-50th of what the survey has to say.

We already operate in an evidence-rich environment, but one that is data-engagement weak. This is not necessarily a bad thing; choosing a university is, and should be, as much a personal, even an emotive, decision as a rational one. But, if the state heads in the direction it is signposting, starting anew, ditching data sources and losing time-series evidence, we may be heading towards a darker shade of pale.  And, when we bemoan this loss in a half-decade’s time, there will be no way to backfill the gap.

John Cater is vice-chancellor of Edge Hill University.

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

question marks PhD study

Selecting the right doctorate is crucial for success. Robert MacIntosh and Kevin O'Gorman share top 10 tips on how to pick a PhD

India, UK, flag

Sir Keith Burnett reflects on what he learned about international students while in India with the UK prime minister

Pencil lying on open diary

Requesting a log of daily activity means that trust between the institution and the scholar has broken down, says Toby Miller

Application for graduate job
Universities producing the most employable graduates have been ranked by companies around the world in the Global University Employability Ranking 2016
Retired academics calculating moves while playing bowls

Lincoln Allison, Eric Thomas and Richard Larschan reflect on the ‘next phase’ of the scholarly life