Use of social media metrics in research evaluation ‘narrow’

Universities too often use social media in a “narrow” way, seeking to boost the impact metrics of their research rather than cultivating long-term public engagement, researchers have warned.

Analysis conducted by Mark Carrigan, lecturer in education at the University of Manchester and Katy Jordan, a research associate in the University of Cambridge’s Faculty of Education, examines impact studies submitted to the UK’s 2014 research excellence framework, which Dr Carrigan described as “the first time that money in higher education actually hinged on claims made about social media”.

The research revealed “a fairly consistent pattern overall of around 25 per cent of case studies mentioning social media”, Dr Carrigan said. Yet this figure rose from 13.1 per cent in the biological and medical sciences to 17.6 per cent in the physical and mathematical sciences, 23.4 per cent in the social sciences and a huge 46.3 per cent in the arts and humanities.

“It is hard to see that data,” reflected Dr Carrigan, “and not speculate that the more secure a discipline is, the less likely it is to reach for social media as a means to generate research impact.”

But the pair’s paper, published in Postdigital Science and Education on 4 November, also found evidence of “platform metrics being cited in a naive and problematic manner”. Relatively few case studies mentioned how social media can be used for “involving participants in research”, while “a surprisingly high proportion simply use social media mentions and metrics as a reflection of traditional scholars’ impact or media appearances”.

What this revealed, suggested Dr Carrigan, was “a culture where social media engagement serves as a proxy for being a publicly engaged academic”. Yet this was “a very narrow and in some ways self-defeating approach”, since it ignored how “much really valuable work using social media is about careful, slow relationship-building, not about promoting yourself to the largest audience possible. It’s about identifying and building connections over time. Encouraging people to become social media celebrities isn’t helpful for that.”

The common stress on the use of social media in individual career-building also neglected what it meant for institutions.

“The use of data in impact evaluation,” Dr Carrigan and Dr Jordan’s paper points out, “loosely couples higher education institutions with the data infrastructures of platform capitalism, with uncertain longer-term consequences”. The corporations involved “enjoy a privileged relationship to that data” and had a business incentive to “lock users into their operation in a number of ways”. Like many other people, Dr Carrigan was concerned by the way “these firms have accumulated an unprecedented degree of social, cultural, political and economic power in a very short space of time”.

The post-pandemic university, the paper concludes, is likely to be “a ‘platform university’ in which our dependence on these infrastructures is ubiquitous, normalised and planned for”. Researchers therefore need to move well beyond questions of “how individuals use technology” and wake up to “a broader institutional transformation: how platformisation, the insertion of platforms as intermediaries into a process, changes the character of the mediated activity and exercises an influence over the organisations in which that mediation takes place”.

matthew.reisz@timeshighereducation.com