Altmetrics: what’s not to like?

1:AM forum on altmetrics weighs pros and cons of tracking research impact via blogs, ‘likes’ and social media traffic

October 2, 2014

Researchers’ creativity could be undermined by too many new metrical approaches to research evaluation, the director of the Wellcome Trust has warned.

Jeremy Farrar told a conference in London last week that so-called altmetrics - research metrics that go beyond traditional citation counts - offered the possibility of “driving change” by better connecting researchers with each other and with the wider community. But he said that the current burden on the research community was “massive”.

“We are in danger of overburdening it with ever more approaches, and it is on the edge of not being able to cope…such that we will destroy [its] creativity and innovation,” he told the 1:AM conference on altmetrics on 25 September.

Altmetrics often involve counting the number of mentions or “likes” of papers on social media. Some commentators have suggested that this could provide an early indication of how many citations a paper is likely to accrue ultimately. But Euan Adie, founder of altmetrics provider Altmetric, admitted that the correlations were “very weak”.

Altmetrics were better suited to assessing the broader social impact of papers, he said, adding that while analysing social media was useful, the movement had to go further.

He said that his firm was also assessing mentions on blogs and review websites such as Faculty of 1000, as well as policy documents - although he said it would take human intervention to ascertain whether the policy had been implemented.

Andrea Michalek, co-founder and president of Plum Analytics, said that her company was also counting comments and reviews, Wikipedia mentions and bookmarks in reference managers.

Adam Dinsmore, evaluation officer at the Wellcome Trust, said that the trust used altmetrics to flag up potentially interesting “narratives” about the influence of research it funded. However, he said, a high altmetric score did not necessarily imply that the research had made a crucial scientific impact, and it was important to consider whose attention papers had caught. For example, the trust searched for mentions of its papers in academic syllabuses and on websites with .ac and .gov domain names.

Liz Philpots, head of research at the Association of Medical Research Charities, said that she used altmetrics to monitor public discussion of funded research, but warned that assessing researchers on the basis of altmetrics would risk judging them “on the basis of their communication skills, not their research skills”.

James Wilsdon, professor of science and democracy at the University of Sussex, is chairing an independent review of the use of metrics in research assessment for the Higher Education Funding Council for England. It has received 152 written responses, which will soon be posted online. Preliminary conclusions will appear in March, followed by a final report in June.

He said that the “incredible dynamism and innovation” in altmetrics was “genuinely exciting for all of us who want to see a creative, outward facing, socially engaged and impactful scientific enterprise”. But he echoed Professor Farrar’s concern that it also had the potential to overburden researchers with “yet more complex systems and tools for them to worry about”.

Times Higher Education free 30-day trial

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Reader's comments (3)

Hang on, Ernesto! So you're saying that although this article mentions 'altmetrics' 19 times in all, it's not a very accurate account of altmetrics? And that "many readers will browse and mainly come across the image and caption--and move on" and hence that readers' browsing habits are not in fact a good way of measuring the intellectual content of a piece of writing. I think you'll find that those are precisely the arguments people use against metrics, 'alt' or otherwise, in the first place.
@gabriel sure. That doesn't make Ernesto's point any less valid though. :) Altmetrics isn't actually about the numbers as an end goal. It just has a misleading name. It covers anything that isn't citations, not just social media. Even if it *did* just cover social media, there's obviously value in looking there for supporting evidence of impact as well as 'traditional' outputs, as Ernesto says and as evidenced by things like the THE article from today I'm seeing being tweeted to the right: This kind of stuff came up at the conference fairly frequently, which is why the caption and image on this (otherwise good) piece are unfortunate. (disclosure: I'm the Euan quoted in the article)
I too am not too sure why THE would chose the image they have as they can be misleading to the content and ideas behind it. For anyone wanting a more in-depth review of the conference, I blogged about it here.