UK university launches responsible metrics guide

New document sets 'benchmark' for others to follow, according to research policy expert

February 23, 2017
boys practise surveying
Source: Getty
Measured response: Loughborough’s new policy on metrics ‘sets a new benchmark’

Loughborough University has launched a formal policy on the responsible use of research metrics.

The statement, which includes 10 elements to consider when evaluating research, looks at how the university can use metrics data sensibly.

James Wilsdon, professor of research policy at the University of Sheffield and chair of the Higher Education Funding Council for England’s review on responsible metrics, called Loughborough’s statement the most comprehensive he had seen to date.

But he added that using the principles in hiring, promotion and evaluation criteria would be a bigger challenge than signing up to the manifesto.

The evaluation of research is increasingly dependent on the use of metrics – data-driven statistics that are used to give an indication of research quality. Commonly used metrics include bibliometrics, such as citation counts and journal impact factors.

Metrics are now often used in funding decisions, the recruitment and promotion of staff and to measure the visibility of research. But critics fear that number crunching is not able to give the full picture of research quality.

Loughborough’s research policy manager, Lizzie Gadd, said: “There are lots of things that affect citation rates, some of which you can normalise for, such as the age, field and type of publication. And some of which you can’t, such as the gender and career stage of the author, the type of research and the newness and size of the discipline.”

There have been several attempts to encourage the research community to use metrics more responsibly, such as the San Francisco Declaration on Research Assessment and the Leiden Manifesto for Research Metrics. These are voluntary codes that universities and funders can follow.

Hefce's independent review, published in 2015, also called for a more responsible use of metrics.

In response to these pressures, Loughborough established in 2016 a small working group of academics and administrators to consider the issues surrounding the use of metrics. The result was the new policy.

The policy is based heavily on the Leiden Manifesto, adapting the key principles for Loughborough's context. The principles include using metrics at the same time as expert assessment, making the collection of data and analytic processes open, transparent and simple, and ensuring that career stage, gender and discipline are taken into account when interpreting metrics.

Professor Wilsdon said: “There have been a series of calls for universities and research funders to adopt more responsible uses of metrics.

“But signing up to manifestos is the easy part: the bigger challenge is to embed such approaches in institutional systems, and in the criteria used for hiring, promotion and evaluation.

“A few universities have taken steps towards this, but Loughborough’s policy is the most comprehensive I’ve seen to date, and sets a new benchmark that I hope other universities will follow."

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

opinion illustration

Eliminating cheating services, even if it were possible, would do nothing to address students’ and universities’ lack of interest in learning, says Stuart Macdonald

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

Female professor

New data show proportion of professors who are women has declined at some institutions

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry