REF: static ranking raises questions about management policies

Performance targets to boost league table position are questioned as analysis shows little change in Russell Group REF order

December 24, 2015
People in maze dressed as Santa Claus
Source: Alamy
Stuck in the middle: just three Russell Group institutions achieved major improvements in their research ranking within the group

Universities hoping to enter the top half of the Russell Group based on research prowess are likely to be disappointed because their position in academia’s unofficial pecking order seldom changes substantially, a study suggests.

In the analysis of how elite universities within the 24-strong group have fared against each other in the research assessment exercise since 1996, Newcastle University’s University and College Union branch found that institutions moved up or down by just one place on average between each assessment cycle.

For instance, Newcastle was ranked joint 19th in the Russell Group in last year’s research excellence framework based on its grade point average compared with 20th in 2008. It was 19th and 21st overall in the 2001 and 1996 assessments respectively.

The University of Liverpool also moved very little over the same 18-year period, fluctuating between 20th and 24th position, the analysis shows.

At the top end, Imperial College London, the London School of Economics and the universities of Oxford and Cambridge held the top four spots in the past four assessments.

When assessed on research power – in which GPA is combined with the number of staff submitted – institutions’ positions within the Russell Group also changed very little between exercises, the study said.

Academics at Newcastle compiled the comparison in response to their university’s new institutional goal of being in the top half of the Russell Group based on its results in the next REF.

It has adopted new “minimum expectations” and “aspirational” targets regarding grant income for research-active academics to help it achieve this end.

Other lower-ranked institutions within the Russell Group have also introduced performance management to improve their research output and league table position.

However, UCU leaders believe that their analysis shows that it is unrealistic to expect Newcastle to make a giant leap up the Russell Group standings given the modest improvements seen previously.

A Newcastle spokesman said that staff are worried about the use of league tables as a measure of staff performance.

“We would love to engage in a collective discussion about how to make Newcastle University better and how to know that is happening – that process has not happened so far,” she said.

Just three Russell Group universities managed to achieve major improvements in their research ranking within the group in recent years, the analysis suggest.

King’s College London jumped from joint 16th place to 6th on GPA and 11th to 6th on research power in the 2014 exercise. Cardiff University achieved a similar GPA improvement, although increased selectivity meant its research power ranking fell.

Queen Mary University of London’s leap from bottom place in the Russell Group to the top half between 2001 and 2008, is also one of the few times that an institution has significantly improved its position, the analysis suggests.

Robert Bowman, head of the School of Mathematics and Physics at Queen’s University Belfast, whose analysis of the REF put its cost at £1 billion, said that the “London effect” might explain some of these improvements.

“There is no doubt there is a massive gravitational pull of research funding to London and southeast England, so how are regional universities going to make inroads on research without access to that level of funding?” he said.

“Can a university expect to jump to a much higher position unless they are part of...big funding initiatives such as the Crick Institute or Northern powerhouse?” he added.

However, while he was “against targets imposed blankly on all staff”, Professor Bowman believed that “aspirational targets” could play their part in improving institutional performance.

“If you are an ambitious academic – and most people are – you will probably aspire to have one or two postdocs, which, with overheads, will cost about £80,000 a year each,” he said.

“If the aspiration is for each academic to raise resources towards their research, an income target north of £100,000 a year becomes obvious.

“If you can get another 25 per cent of staff to get a postdoc, or two, and related funding, then you might take performance to another level.”

A spokeswoman for Newcastle University said that it submitted about 80 per cent of eligible staff to the REF 2014.

It hoped “to increase this number and enter as many eligible staff as possible into the next REF”, while also increasing the average quality of submissions.

“If we achieve both these aims then we would meet our institutional objective of being in the top half of the Russell Group based on our analysis of our performance in the 2014 REF and an assessment of what degree of improvement was realistically achievable by 2021,” she added.

Russell Group rankings and how they’ve changed

Institution REF 2014 ranking by GPA RAE 2008 by GPA RAE 2001 ranking RAE 1996 ranking 2014 rank by research power 2008 rank by research power
Imperial College London 1 4 2 4 8 7
London School of Economics 2 =2 4 3 28 29
University of Oxford 3 =2 3 1 2 1
University of Cambridge 4 1 1 2 3 2
Cardiff University 5 =16 7 10 18 15
King’s College London 6 =16 =15 14 6 11
University College London =7 5 6 5 1 4
University of Warwick =7 7 5 6 15 16
University of Edinburgh =9 9 =12 8 4 5
University of Bristol =9 =11 =12 9 9 10
Queen Mary University of London =9 10 24 22 22 21
University of Sheffield =12 =11 11 12 13 9
University of York =12 8 14 7 23 22
University of Manchester 14 6 8 15 5 3
University of Southampton 15 =11 9 13 11 14
Durham University 16 =11 10 11 20 19
University of Leeds 17 =11 17 17 10 8
University of Glasgow 18 22 18 19 12 13
University of Nottingham =19 18 21 18 7 6
Newcastle University =19 20 19 21 16 17
University of Exeter 21 21 20 23 21 25
University of Birmingham 22 19 =15 16 14 12
University of Liverpool 23 24 22 20 19 18
Queen’s University Belfast 24 23 23 24 17 20


Print headline: Static league table casts doubt on targets

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Reader's comments (4)

National applied research effort in particular, could be usefully deepened and widened across HE by extending the new BIS Trailblazer Apprenticeship scheme to level 8 Doctoral training, and specifically engaging more effectively with SMEs that possess R&D capability, good growth prospects and exporting potential for the UK. Perhaps by reconfiguration of the new mandatory Apprenticeship levy on large business combined with of the current training levies operated by some sectors of UK industry and commerce (needing funding revision any way), plus the Research Councils and other Government Agencies allocated funds for the Doctoral training programme. A competent PhD student domiciled in a good SME working on a company initiated topic, well supported and advised by competent university research supervision, tied in to Innovate UK/Catapult centres resources and skills, might well serve as an appropriate model.
Is there a composite of research excellence and teaching excellence at the undergraduate level? Visiting universities, you sometimes feel that the two are inversely proportional. Great research = crummy teaching, and vice versa. If REF really wanted to encourage the "best" universities it would throw a few research pennies at universities that are good teachers.
Aspirational targets and all that are just a useless and unrealistic thing. In my Dept. these targets are fixed at about 125k/year for Lecturers all the way up to about 1 million/year for Professors. Did any of the admin/management deciding on those targets have a clue about funding resources and success rates in the UK? Let's make it simple for them to understand. In my field, If you consider all research funds that were available in the UK in 2014 (across the board, including private funding not just State money) vs the number of academics that could have potentially applied for these money you end up with an estimate in the region of 10k/year/academic available. If you only consider those academics that applied you have an average of about 35k year income. Further, if you look at how funding was distributed, you will notice that a relatively small number of academics received massive funding vs all others left with peanuts from smaller grants. If the aim is to produce good science and, ultimately, impact, all deserving researchers have to be given appropriate resources and their performances must be measured properly. They should not have to fight for peanuts but they should be accountable for how research money are used.
If Newcastle was serious about improving research, the management would do something about the atrocious SSRs, especially in the humanities departments. Many of these departments have been teaching huge classes for years, while providing disproportionate funding for the rest of the university's activities. It is a testament to the dedication and hard work of the staff that Newcastle continues to do as well as it does in the NSS, let alone the REF, in the face of such serious challenges. When will managements learn that if you threaten academics to try to get them to produce more, they will actually produce less.