UK university launches responsible metrics guide

New document sets 'benchmark' for others to follow, according to research policy expert

February 23, 2017
boys practise surveying
Source: Getty
Measured response: Loughborough’s new policy on metrics ‘sets a new benchmark’

Loughborough University has launched a formal policy on the responsible use of research metrics.

The statement, which includes 10 elements to consider when evaluating research, looks at how the university can use metrics data sensibly.

James Wilsdon, professor of research policy at the University of Sheffield and chair of the Higher Education Funding Council for England’s review on responsible metrics, called Loughborough’s statement the most comprehensive he had seen to date.

But he added that using the principles in hiring, promotion and evaluation criteria would be a bigger challenge than signing up to the manifesto.

The evaluation of research is increasingly dependent on the use of metrics – data-driven statistics that are used to give an indication of research quality. Commonly used metrics include bibliometrics, such as citation counts and journal impact factors.

Metrics are now often used in funding decisions, the recruitment and promotion of staff and to measure the visibility of research. But critics fear that number crunching is not able to give the full picture of research quality.

Loughborough’s research policy manager, Lizzie Gadd, said: “There are lots of things that affect citation rates, some of which you can normalise for, such as the age, field and type of publication. And some of which you can’t, such as the gender and career stage of the author, the type of research and the newness and size of the discipline.”

There have been several attempts to encourage the research community to use metrics more responsibly, such as the San Francisco Declaration on Research Assessment and the Leiden Manifesto for Research Metrics. These are voluntary codes that universities and funders can follow.

Hefce's independent review, published in 2015, also called for a more responsible use of metrics.

In response to these pressures, Loughborough established in 2016 a small working group of academics and administrators to consider the issues surrounding the use of metrics. The result was the new policy.

The policy is based heavily on the Leiden Manifesto, adapting the key principles for Loughborough's context. The principles include using metrics at the same time as expert assessment, making the collection of data and analytic processes open, transparent and simple, and ensuring that career stage, gender and discipline are taken into account when interpreting metrics.

Professor Wilsdon said: “There have been a series of calls for universities and research funders to adopt more responsible uses of metrics.

“But signing up to manifestos is the easy part: the bigger challenge is to embed such approaches in institutional systems, and in the criteria used for hiring, promotion and evaluation.

“A few universities have taken steps towards this, but Loughborough’s policy is the most comprehensive I’ve seen to date, and sets a new benchmark that I hope other universities will follow."

holly.else@tesglobal.com

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Most Commented

men in office with feet on desk. Vintage

Three-quarters of respondents are dissatisfied with the people running their institutions

students use laptops

Researchers say students who use computers score half a grade lower than those who write notes

Canal houses, Amsterdam, Netherlands

All three of England’s for-profit universities owned in Netherlands

Mitch Blunt illustration (23 March 2017)

Without more conservative perspectives in the academy, lawmakers will increasingly ignore and potentially defund social science, says Musa al-Gharbi

A face made of numbers looks over a university campus

From personalising tuition to performance management, the use of data is increasingly driving how institutions operate