REF 2014: winners and losers in 'intensity' ranking

Alternative ranking of REF results maps university performance against the proportion of eligible staff submitted.

December 19, 2014

REF 2014 research intensity ranking - click here

Queen’s University Belfast and Brunel and Loughborough universities are three of the biggest winners when their quality scores in the research excellence framework are weighted for research intensity.

All three universities rank relatively low down among research-intensives when institutions are ranked solely on the basis of the standard grade point average of their REF results (see the methodology here for an explanation of GPA).

But when GPA is multiplied by the fraction of eligible staff submitted by each institution, these three universities rise up at least 30 places, having submitting relatively high proportions of their staff.

Queen’s University Belfast rises highest, from joint 42nd to joint eighth.

Notable institutions to fall away from the upper ranks on this score include Queen Mary University of London, down from joint 11th to joint 34th, and Cardiff University, down from 6th to 50th.

Figures on the proportion of eligible staff submitted are calculated by dividing the number submitted – reported by the Higher Education Funding Council for England – by the number eligible to be submitted – supplied by the Higher Education Statistics Agency.

The latter figures were unavailable when Times Higher Education went to press with its main rankings, which use GPA and ‘research power’ as the primary indicators.

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Reader's comments (11)

Thank you for updating this, I guess. If we really have to play the ranking game, this is the only way to play it. All of the other information based on GPA alone is just propaganda really.
A very helpful update indeed. GPA rankings on their own are pretty meaningless and only help those places who thought they should be "gaming" the scores.. Research Power/Intensity displays a much more meaningful story and should arguably be used exclusively for future rounds when results are being reported. Will the Intensity rankings also be made available at subject/UoA level?
I have just written in expectation of these tables. Will Times Higher Education be able to produce them by Unit of Assessment as well?
Is Queen's University Belfast or Queen Mary University of London depicted as a Queen bee in the photo?
Surely the photo illustrates a typical university administration: a queen bee surrounded by drones! Now that the gaming element is so obvious in many of the REF outcomes, I am surprised it was ever allowed in the first place. But consider the following message from an anomymous head of department to his colleagues, in an anonymous UK university: "It is true that the REF is in principle an "institutional game", which in our case resulted in about two thirds of our staff having been returned. However, I find it important to stress that this is an accomplishment of the whole department, i.e., ALL academic staff, no matter whether or not you have been returned as an individual researcher. The research performance of the department is very much dependent on our long term internal culture, the way we treat, respect and collaborate with each other. And this is exactly what I think we are doing right. Because everyone is part of that, I hope that all of you feel proud of our REF achievement." Now that is true enlightenment. And how rare these days.
On the basis that the GPA figures can and are easily manipulated, why does the TES allow these unrepresentative figures be released for general consumption and use by many for years to come?
This is a much better measure - thanks for publishing it. Please do release it by subject!
Notably most HEIs are only about half a GPA percentage point below the very best, presumably largely achieved on shoestring budgets for many. Perhaps the forthcoming REF funding allocations should thus primarily focus on further improvement of quality, and raising the number of UoAs and eligible staff across the board. To this end possibly the top few units having developed robustness could upgrade to self funding collaborative research institutes, serving as agents for inward FDI to the UK.
If you read the news page of any UK HEI, you will learn that everyone is a winner. Even institutions that dropped down in their ranking by so many points at this round appear to have lots to celebrate. While it sounds odd to see losers celebrate, I guess putting a brave face after a bad game is an achievement by its own. Every HEI played the game to their best advantage …..but the research ranking should ultimate be based on overall impact made by the academic/research community in the institution: not just selected few submitted to get a higher score. What we really needed to know was the number of eligible staff not returned and your updated ranking is just a timely blessing. Is the best way of assessing research in an institution NOT by evaluating all research outputs from all eligible staff? I wonder how many academics are demoralised by not being included and how many will lose out in distribution of research funding based on the game played for REF submission. The number of unreturned academics is staggeringly high and for post-92 universities, the vast majority of academics appear to be not returned. If research funding continues to be polarised both at national and institutional level, it means that a significant number of UK graduates are taught by academic staff that are not counted as researchers. That is really a shame! Dr Solomon Habtemariam
There is a very simple explanation for movements such as Queens University (upwards) from joint 42nd to joint 8th, and Cardiff (downwards) from 6th to 50th. Both measures (GPA and GPA multiplied by a ratio) are meaningless. Simple arithmetic is inadmissable in respect of the REF's four level scale because that scale is merely ordinal (at best). Moreover, if one reads the first chapter of Alasdair MacIntyre's "After Virtue," one can see that the REF ordinal scale cannot be approximated to an underlying "quantifiable" interval scale. Incredibly, the missing ingredient is teaching (see Andrew Warwick's "Masters of Theory: Cambridge and the rise of Mathematical Physics") . This argument is set out in greater depth at Or Google "Lies Damn Lies and the Research Excellence Framework"
Thank you for that comment Stephen Elliot, it needed to be said. Even accepting some sort of numerical scale can be applied I have been slightly puzzled by the assumption that the grade point averages are calculated in a linear way instead of by the non-linear funding formula.

Have your say

Log in or register to post comments

Most Commented

Home secretary says government will support 'best' universities

Man handing microphone to audience member

Academic attainment of disadvantaged students can be improved if they can decide how they are assessed, study claims

Woman drinking tea from saucer

Plugging a multibillion-pound deficit exacerbated by June’s poll result may require ‘drastic measures’, analysts have warned

Italy's gold medallist

New measures to ensure universities are ‘not penalised’ for taking poorer students also outlined for next stage of TEF

Classroom, school

Higher education institutions can and should do more to influence education at a secondary school level, says Edward Peck