Judging REF 2014 was ‘inspiring and humbling’

The exercise was robust and positive, says Willy Maley, who found work on the English subpanel to be like ‘a well-run exam board’

January 1, 2015

Source: Patrick Welham

The REF is not responsible for the management strategies of individual institutions or particular responses to research assessment

Four years ago I was invited to join the English subpanel for the research excellence framework 2014.

I accepted. Having just run a “mini-REF” in my own department, I wanted to be involved in the real thing, seeing participation in the process of peer review as a vital service to the discipline. Here was an opportunity to immerse myself in the work of colleagues across the sector and to acquaint myself with the workings of other higher education institutions. I see virtue in making comparisons and identifying best practice. Research assessment has been a part of the academic landscape since I was appointed at Goldsmiths, University of London in 1992 as a temporary lecturer. I took part in a pilot for the 1996 research assessment exercise there, and as an early career researcher found that exercise both chastening and useful.

After experiencing the “churn” of restructuring, I saw in the REF something positive. An expert peer review process with panels composed of senior academics with considerable experience across the spectrum of research carried out in my discipline struck me as a sensible way to judge the quality of work being produced. The REF builds on its predecessor, the RAE, and for my subject in ways I found encouraging. As someone who had pioneered creative writing at Goldsmiths and launched the master’s in that subject at the University of Glasgow, I was heartened by the inclusive criteria that the panel established as the framework for assessment. Work in the fullest range of forms and formats would be eligible. Creative, co-authored, collaborative and interdisciplinary work would take its place alongside more familiar types of research.

Yet I was unsurprised by the negative reaction from some quarters. These are not good times for higher education. A climate of change and cuts is not conducive to collegiality or to trust in processes that, sound in themselves, may be used for purposes other than those for which they were intended. The REF is not responsible for the management strategies of individual institutions or particular responses to research assessment.

I feel privileged to have been part of what one fellow panellist characterised as “a well-run exam board”. We adhered strictly to the published criteria, despite suggestions that there would be nods and winks behind the scenes. The experience was inspiring and humbling in equal measure: mind-expanding in the way that reading intensely over a nine-month period the finest work in one’s field is bound to be; ego-shrinking in that I emerged with a more modest idea of my own achievements, and a strong sense of the range and richness of work in my subject area.

Some cast doubt on the capacity of the panel to have read with sufficient care the work submitted. I can vouch for the commitment of the panel on which I served. Outside the REF, I have PDFs for breakfast, so while this was the most demanding period of reading I have undertaken in my career it was familiar territory. Impact was new, but there too attention to detail was unflagging, as was adherence to published criteria. Many criticisms of REF resort to assumptions that would be dispelled by careful scrutiny of the documentation.

According to Derek Sayer, professor of cultural history at Lancaster University, “the REF casts a long shadow over British academic life” (“Protest for a REFormation”, 11 December 2014). REF 2014 – the first research excellence framework, and the first such exercise to assess “impact” – entailed hundreds of academic colleagues carefully calibrating the quality of work and workplace of thousands of other colleagues and, assisted by dozens of expert impact assessors, examining the effects of that work on the world beyond the academy. One scholar on a crusade, or some scholars confusing shadows cast from other obstacles to academic freedom with a process of peer review that’s been with us for nearly 30 years, cannot compete with the comprehensive experience and expertise of panels put together by the Higher Education Funding Council for England. As academics, we are all players, and all referees, but above everything, we are all colleagues.

Quixotic crusades make good headlines. Journalistic conclusions come easy. The hard work of assessment takes time. I do not accept Sayer’s characterisation of the REF: “What began as a ‘light-touch’ periodic audit in 1986 has spawned university bureaucracies that continually monitor and seek to manage individuals’ research within REF priorities and timelines.” I have issues with the new managerialism, with the risks to academic freedom posed by changes in university governance, and the shift from collegial to corporate models. I see long shadows cast by cutbacks and privatisation, but I have complete confidence in the REF, as I had in the RAE.

Metrics might suit me personally – I have quantity of output in spades – but if I envisage the process that would best support work done in my field in all its variety and vibrancy then it would be the exercise in which I have just participated. My experience of REF 2014 was indeed of a well-run exam board, one that took a generous view of the work that colleagues do, the circumstances in which they do it, and the impacts it is having, and arrived at robust judgements.

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (1)

Ever read Foucault, m8?

Have your say

Log in or register to post comments

Featured Jobs

Professor-Keith Cameron Chair of Australian History UNIVERSITY COLLEGE DUBLIN (UCD)
Senior Procurement Officer UNIVERSITY OF THE WEST OF SCOTLAND
Clinician, Small Animal Emergency Services UNIVERSITY COLLEGE DUBLIN (UCD)
Director COVENTRY UNIVERSITY COLLEGE

Most Commented

question marks PhD study

Selecting the right doctorate is crucial for success. Robert MacIntosh and Kevin O'Gorman share top 10 tips on how to pick a PhD

India, UK, flag

Sir Keith Burnett reflects on what he learned about international students while in India with the UK prime minister

Pencil lying on open diary

Requesting a log of daily activity means that trust between the institution and the scholar has broken down, says Toby Miller

Construction workers erecting barriers

Directly linking non-EU recruitment to award levels in teaching assessment has also been under consideration, sources suggest

Microlight pilot flies with flock of cranes

Reports of UK-based researchers already thinking of moving overseas after Brexit vote