Tread carefully with altmetrics, European Commission told

Transparency ‘crucial’ to the introduction of new metrics in Continent’s research system

March 31, 2017
Man crossing gorge

Alternative metrics should be used by the European Commission alongside expert judgement and other measures of research quality, according to a new report.

The report cautions against relying too heavily on new ways of measuring research when developing the open science agenda in Europe.

It adds that ensuring that data are collected and processed transparently will be crucial to the process of introducing new research measurement indicators.

Carlos Moedas, the European Commissioner for research, science and innovation, has made opening up science in Europe a key objective of his tenure. By doing so, he hopes to make research more accessible to society, and to make the products of research, such as papers and data, available to other scientists as soon as possible.

As part of this, he commissioned an expert advisory group to look at the strengths and weaknesses of alternative metrics. Altmetrics include measurements of social media activity, for example, and can be used to give a broader picture of research impact compared with more traditional measurements such as citation counts and journal impact factors.

The group, led by James Wilsdon, professor of research policy at the University of Sheffield, came to its conclusions by reviewing the literature and evidence submitted to it about how new metrics could help to advance the work on opening up European science.

It found that some alternative indicators of open science are already available. These include measures of document views and download numbers, co-authorship counts and the data on the number of tweets, likes and shares on social media.

But the report, Next-Generation Metrics: Responsible Metrics and Evaluation for Open Science, cautions that these are “not perfect”. “Many available altmetrics are not yet robust or ready to be used for purposes of research evaluation and need to be monitored carefully before they are applied in such contexts,” the authors say.

“Current indicators need to be improved, and should co-exist, with additional indicators and peer review, in the policy and evaluation process,” they add.

Professor Wilsdon told Times Higher Education that the advisory group was not “cheerleading” for alternative metrics. But he added that many of these indicators try to capture information about the broader audience and impact of research, which is a “good thing”.

The European Commission is interested in gaining a more holistic picture of research because it is under pressure to demonstrate the value of research funding. “There is a quite a lot of scrutiny of and pressure on EU research budgets,” Professor Wilsdon added.

Professor Wilsdon said that the Commission should avoid linking metrics to the allocation of funding because this gives people “incentives to game them”.

Instead, metrics should be used “in intelligent ways to measure what goes on in research once it has been funded”, he continued.

The report adds that an open, transparent and linked data infrastructure will be needed if next-generation metrics are to be used in research evaluation. “How underlying data are collected and processed is crucial,” the group says.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

Female professor

New data show proportion of professors who are women has declined at some institutions

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry

But the highest value UK spin-off companies mainly come from research-intensive universities, latest figures show