NSS and DLHE reform: don’t throw the baby out with the bathwater

John Cater argues that higher education would be poorer for the loss of student survey data

March 7, 2016
Survey responses

The phones have been ringing for the past month and, across the country, four-fifths of last summer’s graduates have answered the Destinations of Leavers from Higher Education survey, as they have for the past decade. And, across the campus, almost three-quarters of this year’s finalists are answering at least 23 questions about their academic experience in the National Student Survey. A dozen years, almost two dozen questions, five choices and more than 250,000 respondents every spring. 

So we now know what the vast majority of graduates are doing six months into their post-graduation lives, and we will soon know what the cohort behind them think of their university experience. Yet, we decry, both studies and are in grave danger of dumping a decade or more of time-series data. Why?

As academics we’re critics, bred to focus on limitations and shortcomings. An evening with us is like a date with an actuary. But where else will you find such a welter of information provided by those with real experiential insight? Of course, the DLHE isn’t perfect; not everyone’s long-term career is going to be determined by what they are doing in the first winter of discontent. But it gives us a vital indicator of the diminishing number affording postgraduate study, of the initial earnings of this April’s taxpayers, and of the first steps in the labour market. Yes, it should be supplemented by time-series data that follows the individual, not just the sector, but this is not an argument for its disappearance.  

The future of the NSS has also been subject to review; it will be different and it may well also disappear, at least in its present form. We already know that question 23 on students’ unions – a vital measure of student engagement – has been consigned to the dustbin of history. The Higher Education Funding Council for England is warning institutions and departments against exerting undue influence on this year’s NSS respondents, having identified abuses in the past. But reported instances have been rare and are an argument for a different approach to data collection focused on the individual, not the institution – not data dismissal.

Basic questionnaire techniques, such as asking some questions in the negative, would help to eliminate the column-tickers, those who return 22 unthinking or uncritical “fives” or, more rarely, 22 unthinking or malevolent “ones”. Or, as an alternative, eliminate the outliers from the calculation of institutional means so beloved of some league table compilers. And persuade other compilers to stop using two categories (scores of four and five) from one question (overall satisfaction) and representing this as a holistic analysis of anything more than one-50th of what the survey has to say.

We already operate in an evidence-rich environment, but one that is data-engagement weak. This is not necessarily a bad thing; choosing a university is, and should be, as much a personal, even an emotive, decision as a rational one. But, if the state heads in the direction it is signposting, starting anew, ditching data sources and losing time-series evidence, we may be heading towards a darker shade of pale.  And, when we bemoan this loss in a half-decade’s time, there will be no way to backfill the gap.

John Cater is vice-chancellor of Edge Hill University.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

Female professor

New data show proportion of professors who are women has declined at some institutions

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry