UK university launches responsible metrics guide

New document sets 'benchmark' for others to follow, according to research policy expert

February 23, 2017
boys practise surveying
Source: Getty
Measured response: Loughborough’s new policy on metrics ‘sets a new benchmark’

Loughborough University has launched a formal policy on the responsible use of research metrics.

The statement, which includes 10 elements to consider when evaluating research, looks at how the university can use metrics data sensibly.

James Wilsdon, professor of research policy at the University of Sheffield and chair of the Higher Education Funding Council for England’s review on responsible metrics, called Loughborough’s statement the most comprehensive he had seen to date.

But he added that using the principles in hiring, promotion and evaluation criteria would be a bigger challenge than signing up to the manifesto.

The evaluation of research is increasingly dependent on the use of metrics – data-driven statistics that are used to give an indication of research quality. Commonly used metrics include bibliometrics, such as citation counts and journal impact factors.

Metrics are now often used in funding decisions, the recruitment and promotion of staff and to measure the visibility of research. But critics fear that number crunching is not able to give the full picture of research quality.

Loughborough’s research policy manager, Lizzie Gadd, said: “There are lots of things that affect citation rates, some of which you can normalise for, such as the age, field and type of publication. And some of which you can’t, such as the gender and career stage of the author, the type of research and the newness and size of the discipline.”

There have been several attempts to encourage the research community to use metrics more responsibly, such as the San Francisco Declaration on Research Assessment and the Leiden Manifesto for Research Metrics. These are voluntary codes that universities and funders can follow.

Hefce's independent review, published in 2015, also called for a more responsible use of metrics.

In response to these pressures, Loughborough established in 2016 a small working group of academics and administrators to consider the issues surrounding the use of metrics. The result was the new policy.

The policy is based heavily on the Leiden Manifesto, adapting the key principles for Loughborough's context. The principles include using metrics at the same time as expert assessment, making the collection of data and analytic processes open, transparent and simple, and ensuring that career stage, gender and discipline are taken into account when interpreting metrics.

Professor Wilsdon said: “There have been a series of calls for universities and research funders to adopt more responsible uses of metrics.

“But signing up to manifestos is the easy part: the bigger challenge is to embed such approaches in institutional systems, and in the criteria used for hiring, promotion and evaluation.

“A few universities have taken steps towards this, but Loughborough’s policy is the most comprehensive I’ve seen to date, and sets a new benchmark that I hope other universities will follow."

holly.else@tesglobal.com

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Worried man wiping forehead
Two academics explain how to beat some of the typical anxieties associated with a doctoral degree

Felipe Fernández-Armesto takes issue with a claim that the EU has been playing the sovereignty card in Brexit negotiations

A group of flamingos and a Marabou stork

A right-wing philosopher in Texas tells John Gill how a minority of students can shut down debates and intimidate lecturers – and why he backs Trump

A face made of numbers looks over a university campus

From personalising tuition to performance management, the use of data is increasingly driving how institutions operate

Kenny Dalglish

Agnes Bäker and Amanda Goodall have found that academics who are happiest at work have a head of department who is a distinguished researcher. How can such people be encouraged into management?