Reflections on the REF

The Research Excellence Framework, like losing hair and acquiring an encyclopedic knowledge of acronyms, is a yardstick with which to measure one’s journey in UK higher education.

My first was in 2008 (OK, technically an RAE), my second in 2014, which I assumed at the time would also be my last (assessment frameworks may go on and on, but editors tend not to).

Here we are eight years later, though, getting to grips with another set of REF results, and the divvying-up of billions of pounds in quality-related (QR) research funding, not to mention bragging rights.

The passing of time has, however, brought changes to the REF. So is this latest exercise better or worse than the previous one? And what comes next?

For David Price, outgoing vice-provost (research) at UCL, the positives include a more joined-up approach across the main panels and the fact that all research-active staff had to be submitted this time (a recommendation made by Lord Stern in his eponymous review in 2016).

The game-playing that was such a feature of REF 2014 was “not noticeable”, Price said, and if the new rules about universal submission led to a rise in teaching-only contracts in some institutions, then that could be seen as “resolving contractual ambiguities” rather than anything untoward.

Weighed against this, for Price, were the challenges that had arisen during the REF assessment process because of Covid, while he argued that institutional environment statements were “of little value”, and observed that because outputs represented only a small fraction of UK activity in many units of assessment, it was difficult to draw conclusions from REF 2021 about the “health of the nation”.

Another question is whether the REF is too onerous and bureaucratic an exercise.

James Wilsdon, director of the Research on Research Institute, pointed to a 2021 survey of researchers’ attitudes towards the REF, which found that while the burden on individuals had reduced, for institutions it had, if anything, got worse.

He regretted that the recommendations of the Stern review had been partially rather than fully implemented: “We got a watered-down version, thanks to pushback from Russell Group universities, which have the biggest vested interest in maintaining the status quo. Plus, as always happens, some individual institutions over-interpreted or policed the implementation, creating a more rigid, inflexible or brutal process at a local level than intended.”

One of the changes, in comparison with 2014, is that the REF is no longer the only game in town.

There is the Teaching Excellence Framework and the Knowledge Exchange Framework; university rankings track performance and bestow prestige; and while increases in R&D support are promised, the government tends to favour specific funding, rather than QR, over which it has less control.

So what might the future hold for this central piece of the UK research machinery?

Gemma Derrick, associate professor of higher education at the University of Bristol, says that, on balance, “a world with REF is still better than a world without REF”.

However, in her view, “the REF as a tool has suffered from a loss of purpose”, and it is time to harness its power in new ways, to “re-purpose the REF towards incentivising desirable research behaviour, rather than simply assessing past performance”.

Both Price and Wilsdon say that as long as QR exists as a strand of research funding, then the REF or some reincarnation of it will be necessary – not least because, in Price’s words, “the Treasury will require some audit process that the money is well spent – REF provides that assurance”.

But Wilsdon senses that “the stars are now aligning to support a more radical overhaul of the exercise than at any point in the last 25 years”.

The Future Research Assessment Programme, set up by Research England in 2020 to review the REF and wider research system, is expected to report towards the end of this year.  When it does, says Wilsdon, “it will be up to ministers to decide how radical they want to be”.

Perhaps that is the key point: like it or not, the REF is really all about politics. Politicians are contractually obliged to hate bureaucracy, and might therefore take a dim view of such an exercise, which came with a £250 million price tag last time around.

On the other hand, they like accountability (for others), and they like mechanisms via which they can assert at least some control – for example, incentivising impact.

For as long as it serves a political purpose and defends QR, the REF is probably worth it.

john.gill@timeshighereducation.com