REF’s experiment with research culture was always doomed

Just over four months ago universities woke up to a reversal in one of the Research Excellence Framework (REF)’s most ambitious proposals for 2029: to assess UK universities partly on the basis of their research cultures.

This about-turn, announced on 10 December, followed a Research England pilot study of the ways research culture would be assessed and measured. The pilot revealed that while assessing research culture at scale still has merit, how to do this consistently within and across institutions is a major stumbling block. Accordingly, the “people, culture and environment” element of REF 2029 had been rejigged and renamed “strategy, people and research environment”.

As anthropologists, we have been conducting ethnographic research since 2022 on how research culture is understood and experienced within the intricate ecosystem of universities. The REF’s reversal does not come as a surprise to us because, as we have found, formal efforts to foster positive research cultures focus heavily on institution-wide processes and policies, which sit awkwardly alongside – and are sometimes in friction with – the everyday realities of research culture.

These everyday realities – in essence, the cultures that shape the experience and outputs of research – are made up of a varied tapestry of relations and local dynamics between researchers and other university staff. But this “relational” take on research culture is quite distinct from the “processual” take, which focuses on the institutional processes through which research culture is formally imagined, managed and evaluated.

Of course, clear and transparent processes and structures are indispensable to fostering a positive research culture. However, the fact remains that the relationships that actually make up these cultures are the lifeblood of research.

Take the experience of Andrew, a junior lecturer in a particular department of a Russell Group university. Before taking up this post, Andrew gave a seminar presentation at the same department. The paper was well received and when a lectureship opened several months later, he applied and eventually secured the job. On reflection, Andrew believes his memorable paper helped him stand out among the longlisted applicants (chosen with reference to essential criteria in the job description) and make it on to the shortlist.

Now, a process-focused understanding of research culture would likely raise concerns about this case. How would the hiring committee’s putative perceptions of Andrew’s seminar paper align with principles of fair recruitment? If Andrew’s surmise is correct, is this not an example of undue bias? These are legitimate and important concerns.

However, what this way of viewing research culture fails to take into account is the inherently relational quality of research, which seminars express and foster as focal social events in departmental life. For early-career researchers in particular, they offer opportunities to break free from institutional hierarchies, be recognised beyond immediate networks, and leave impressions that may, indeed, later influence career trajectories. Alongside rigorous, fair and transparent processes, academic hiring unfolds within a social landscape shaped by these relational encounters.

How, then, might the distance between these on-the-ground social dynamics and the often standardising processes that are used to manage “Researcher Culture” (emphasis on upper-case R and C) be bridged?

As an example of the complexities involved, take the “Research Culture Report” (RCR), an interactive data visualisation platform created by one university in our study in order to make research culture within the institution visible to university management. The idea was to present “the word on the street in a structured way with colours”, according to someone involved in its design. It presents highly condensed summaries of academics’ “lived experience” (taken from survey data) alongside various quantitative indicators to provide a snapshot of local research cultures across the institution and to support institution-wide interventions.

In our ethnography of the RCR’s creation and implementation, however, we found divergent expectations between the tool’s users and producers. Some of the staff (including academics) who fed into it felt it condensed complex qualitative findings into generic-sounding policy points lacking sufficient detail to be actionable, such as “there are concerns over fairness and transparency of promotions”. On the other hand, staff developing the RCR felt that this condensation was necessary for it to be effective as a tool for gauging research culture “at a glance” and driving improvement in a strategic, institutionally consistent way.

This exemplifies a productive tension between relational and processual perspectives. Both parties recognise and are responding to “relational” (social, cultural) issues (such as recruitment practices, team dynamics, staff well-being and collegiality). But processual logics rest on the assumption that everything of importance should be measured (and measurable). In reality, the relational characteristics of research culture are remarkably complex to measure (if not at least partly immeasurable).

So how might the desire to improve research culture be heeded while avoiding the impasse that the REF’s focus on processes has led to? Our suggestion is threefold.

First, effective interventions in research culture need to recognise that the problem is not solely one of devising processes, strategies or measurements. Crucially, the task is also to understand how those interventions articulate with the relational dynamics of cultures of research on the ground. This involves iterative, context-specific engagement with researchers. Institution- or sector-wide frameworks or measurements are a poor substitute for this.

Second, bridging the gap between processes and relations requires institutions to work at the problem from both ends. The work on developing road maps, metrics, training programmes, best practices and so on should be complemented by the work of resourcing grassroots and localised initiatives led by researchers themselves that foster the relationships that make research communities work best.

Last, as anthropologists we would point out that ethnographic research methods – available to many universities through their staff and students – are a powerful tool for engaging with the relational fabric of research culture. What makes ethnography unique is precisely its relational quality – understanding social and cultural dynamics by engaging with them. Ethnographic methods can provide institutions with a robust evidence base that probes the context-specific relationships that make research work.

The difficulties REF ran into with its processual approach to research culture should be a wake-up call. Research culture still matters, but to foster it, universities must take heed of its inherently relational character and avail themselves of the relational tools needed to cultivate and improve it.  

Martin Holbraad, Dan Nightingale and Aeron O’Connor are social anthropologists at UCL.