The REF games are even more brutal this time around

Forcing academics on to teaching-only contracts based on flawed assessments of their research is ruining careers, an anonymous academic says 

January 3, 2019
Illustration: THE main opinion 3 Jan 2019 issue
Source: Miles Cole

In theory, requiring UK universities to submit all of their research-active staff to the 2021 research excellence framework is a great innovation. In previous iterations of the exercise, when universities could choose who to submit, the results were marred by so-called game playing, whereby certain institutions submitted much smaller proportions of their eligible staff than others did, in order to maximise their quality scores.

But those who run the REF do not seem to understand how deeply game playing is etched into the psyches of modern universities.

Instead of creating a fairer assessment and reducing the burden on everyone of deciding who to submit, it is becoming apparent that the new rules are leading to even more ruthless and deceitful behaviour on the part of university managers.

Since all staff on teaching and research contracts must submit at least one “output” to the 2021 REF, those who do not have at least one deemed 3* (“internationally excellent”) or 4* (“world-leading”) are in grave peril. Last month, Times Higher Education revealed that the Russell Group alone spent nearly £50 million on severance payouts during the 2017-18 financial year: more than 50 per cent more than during the previous year.

The alternative to making people redundant is to force them on to teaching-only contracts. This is what has happened to me. I am a Russell Group academic with research grants and numerous publications to my name, along with extensive PhD supervision and teaching experience. But after I was deemed to have no outputs better than 2* (“internationally recognised”), I was hauled before my line manager and, in effect, told – without prior warning – that unless I produce a 3* or 4* article within months, my contractual responsibility for research will be removed.

While teaching-only contracts may be ideal for some people, those conducting research see them as an effective demotion. Nor am I the only one to have been subjected to such threats. It has also happened to senior colleagues – and I have heard similar tales from other institutions. Moreover, this contractual game playing is only likely to snowball over the next two years as the REF submission date approaches.

The demand to produce a 3* output at the drop of a hat is, of course, completely unrealistic. Planning new research, obtaining funding, conducting the research, writing up and finding a publisher can take years. We have been told that we will be also able to stay on teaching and research contracts if we can find another unit of assessment that takes a rosier view of the quality of our existing outputs and is prepared to submit us. But that option is not open to the colleague who was lured on to a teaching-only contract with the promise of a promotion that never actually materialised.

You can imagine what all this has done to the relationship between management and staff in my department. To make matters worse, the “evaluation” of our publications was performed by a single individual who, in many cases, has little or no knowledge of their context. Most of my departmental colleagues use different methods from mine: with the best will in the world, this makes it difficult for them to accurately judge the quality of my articles.

I do not deny that judgement of academics according to universal standards can be legitimate. But such a system needs a high level of reliability and validity. It needs to call on the expertise of those who know something about the research area in question. Teaching assessment, for instance, is carried out by at least two people from different subject backgrounds, and their assessment is blind, minimising the risk of bias.

Of course, my bullied colleagues and I can bring the union in, and I am sure the reps will do their best. Unfortunately, however, the UK’s abolition of tenure in the 1980s, combined with the massive financial and legal power of the universities, means that, realistically, if we do not accept the new contracts then we will probably find ourselves joining those already made redundant. Even to complain is a big risk: that is why I am writing anonymously.

In the past, universities’ desperation to win the REF game created a frenzied transfer market in which the big boys and girls with a string of 4* outputs were able to command considerable fees. But while not being selected for submission was never good career news, this time around, those marked down in internal assessments face a sudden and permanent end to their research careers.

Such brutal treatment of effective academics will do nothing to improve the accuracy of the REF as an assessment of the relative strengths of UK universities’ research. It will do nothing to drive a fairer distribution of more than £1 billion a year in research funding. But it will do further damage to the strength of the research on which the government is relying to power the UK’s knowledge economy post-Brexit – just as we are beginning to experience a significant brain drain precisely on account of Brexit.

In short, if universities can’t stop their game playing, the whole country risks being cheated.

The author wishes to remain anonymous.

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Related articles

Reader's comments (15)

I'm not overly sympathetic to the author. In my discipline, something like 60% of all submitted outputs were rated 3- or 4-star in REF 2014. A 3-star publication is therefore hardly an excellent one, merely a solid one. If an academic on a research and teaching contract cannot produce one solid publication in 5+ years, then they should make room so that one of the many bright and unemployed PhDs can have a chance. Moving to a teaching-only contract seems a fair outcome.
@acemoglu It takes two hands to tango. What if the situation was due to mismanagement from the employer (i.e., the university)? How would that factor into this? Is shifting the blame all on the academic a fair (REF) game?
Come and work at a non-research intensive. We'd welcome you and your outputs with open arms!
I appreciate the position and argument from the anonymous research active academic in last week’s issue (‘The Ref games are even more brutal this time around’) who may be forced by university managers onto a teaching only contract because of a lack of at least one 3* output. I recognise particularly the, to some extent unintended, consequences of the REF in concentrating research and narrowing its scope. However, at least three points seem pertinent from my perspective as a research active academic in a university of applied science (often referred to as ‘post 1992’) where the focus of academics is on knowledge exchange and teaching as much as on research. First, the impact of REF game playing is likely to be differentiated across the diversity of academics, so women, ethnic minority origin and working class academics beware! It was noticeable that the illustration accompanying the article last week showed one white female and seven white male academics being metaphorically cut down to teaching only. Second, for some academics, including many of my colleagues in professional fields who may not (yet) have a doctorate, a teaching only contract may be a welcome relief. Third, and in tension with the second point, no matter what your field, your contract, your institutional context, or the response of your university managers to the REF, as an academic there remains a sector wider pressure to be active in research or advanced scholarship, to publish or perish is simply part of the territory. In a UK survey based study of academics in Nursing, Midwifery and Allied Health Professions, we found that many were struggling to develop their research work and researcher identity. A significant proportion, even in research intensive universities, were subverting researcher identity – they were doing ‘just enough’ to keep managers happy then focusing on teaching, knowledge exchange or leadership. It does seem perverse for managers to deliberately move an academic towards more teaching without considering their effectiveness as a teacher, but on the other hand asking them to juggle multiple identities all at the highest level may not be realistic. REF and TEF game playing at institutional and individual level is just part of the marketization of education policy framework that has reinforced an elite education system, university managers need to understand this wider context and be proactive within it. At a graduation ceremony before Christmas I was dismayed to hear the vice chancellor of a research intensive university boast to the assembled parents and students that his was an ‘elite’ university. I would suggest that his world view rests on the myth of meritocracy and I only hope that his Christmas stocking included copies of Danny Dorling’s ‘Injustice: Why Social Inequality Still Persists’ or Diane Reay’s ‘Miseducation’.
“Mock” refs scores have huge variance. I had two papers scored by four people last time (two internal, two external). The scores on both? 1,2,3,4 stars. One external gave two ones, the other two fours. Both profs at Russell group unis in top ranked departments. Clearly my work divides opinion, but to determine someone’s career trajectory based on one score is grossly unfair
The unsympathetic comment above appears to be going on a level playing field notion that may exist at their place of employment. The problem may not be the production producing a 3* publication but in a hostile work environment the issue can be getting it recognised as one. It should be straightforward but ranking is a subjective process that is open to abuse. I know from experience of having a paper rated as 2* by the internal process and 3/4* by an academic that sat on the REF panel that it is not a precise science. If the department is attempting to portray a particular image/specialism it can influence judgement. Giving early career academics a shot is fine but when you add that to the current move towards "X-Factor" teaching where youthful enthusiasm ticks boxes the process can mask ageism. Moving older academics to teaching only may seem fair but often it is a humiliating demotion calculated to move someone closer to the exit.
"but in a hostile work environment the issue can be getting it recognised as one. It should be straightforward but ranking is a subjective process that is open to abuse". Absolutely valid point. Often times there is no clear oversight of this whole process and no proper feedback. In one instance the feedback for a 3* rated paper was "it dosent meet the criteria for 4*". It is like saying to a student, you have not got a disctinction because you have only got a merit! A lot of good that feedback is!
I agree with the article but for a different reason. Teaching or education contracts should be for those who excel in teaching and improving the quality of education at an institution. These contracts are demeaned if they are used for people who can't do something rather than people who can do something. Plumbers and electricians often work together. You would not make an electrician a plumber because they blew a fuse. You should encourage and develop good electricians and good plumbers, and some excellent individuals who can excel at both. Research and teaching are complementary, but not the same. Research and teaching are both important, and should not be played-off against each other.
I agree with simonkent no place in teaching for failed researchers, just great teachers. And while I understand the point about fair mock REF assessments, someone on a substantial research contract with PhD students and external funding should be able to produce one 3* paper in 5 years. If they're on a full research contract and not produced at least two they should be looking for a new job.
There is nothing necessarily bad about being on a Teaching & Scholarship contract so long as it is voluntary and there is true parity of esteem between teaching and research, including a recognised promotion route; indeed, I switched a few years ago as I near the end of my career and suffered a series of, frankly, gutting paper rejections and revise and resubmits I no longer had an appetite for. However, there is a world of difference between aggregate measures (such as @acemoglu's 60% of submitted REF2014 outputs being rated 3*/4*) that might - just might, despite all the well known assessment flaws and consequent margins for error - be useful for assessing groups of researchers, departments and universities, i.e. the stated purpose of REF, and using such dubious metrics for forced, top-down "performance management" at an individual level, especially when there is a lack of accountability and transparency and no right of appeal. This is just another sad example of the enormous waste of resources represented by REF and TEF and the inevitable gaming of them both. The only good thing about them is that now there is a parity of garbage on both sides of the research/teaching equation.
Nothing wrong with a teaching appointment in a university. They pay the same and you can become a professor. (In my experience, they teach less than most academics). If staff want to pursue research they should be given support and permitted time to plan this. You cant create quality research in a short period of time since the cycle is generally about a year. If you havent done much research in a while though, it is very difficult to reclaim these skills, especially if you are in a discipline where research is done through an intermediary (eg PhD student, postdoc etc).
Firstly the student goes to a university pays an extraordinary amount of money for excellent teaching. How can that teaching be excellent when the teachers' English is so poor as to be unfathomable. The account seen above suggests something is seriously wrong with the core activity of a university, that is its teachers and those teachers' skills to teach. Knowledge is nothing if it stays in the head of someone mumbling at the lectern. The university in this country has lost its way no longer fit for purpose. Researchers need to be in a research institute somewhere, only guest lectering if they meet the high skill standards required to teach. The idea that research, often the pet project of who ever is in charge, stands above teaching high paying clients is perverse.
Excellent comment - I agree wholeheartedly with this sentiment!
Now what we have been told is that for REF 2021, only 'independent' researchers will have to be submitted, so for Research Only staff, that is going to be taken to include only Senior Research Fellows, not postdocs.
The major issue that in many cases, an "internal REF" is used to determine whether a paper is 3 or 4 star. Many papers that well cited and often times practically useful may not fit the category of 3 or 4 because it does nto appear in the right journal (even thought the claim is that the journal dosent matter!) Often times it is unclear who is part of the panel that makes this evalaution, what checks and balances are in place to ensure that the process is fair and free? When these are undertaken by people outside one's home university, how will the home university make sure that those people do not have any unconcious biases that they bring to their evaluations? One needs to ask if there are places where these exercises actually are done by departments or schools without clear and transparent university oversight?