Citation pitfalls ahead

March 6, 2008

Charles Oppenheim (Letters, 21 February) says previous studies "have shown excellent correlations between research assessment exercise scores and citation studies and therefore indicate citation analysis would not greatly affect overall order of RAE score". But such correlations are consistent with individual outliers. A demonstrable correlation is no guarantee that citation metrics will reliably replicate rankings based on quality judgments for individual department. Since the point of the exercise is to allocate funding to individual departments, such discrepancies should not be glossed over.

There is a deeper problem. Quality of research is not the only factor influencing frequency of citation. In my discipline, some topics receive far more attention than others: for example, publications on Homer are more than 30 times more numerous than those on Heliodorus. So if you want to maximise citations, give up on Heliodorus - especially if you are engaged in research with a long-term payoff. Citations accumulate slowly in Classics, so you will need to focus efforts on provocative interventions in current debates to stand any chance of gaining citations that will influence the next funding round.

A funding regime that selectively rewards quick-return contributions on a narrow range of intensively discussed topics will drive high-quality, low-citation research to extinction. That would do wonders for the correlation between quality judgments and citation metrics, but the consequences for the health of the discipline would be horrendous.

Touting correlations without understanding the causal relations that underpin them is a perfect recipe for creating perverse incentives.

Malcolm Heath, Professor of Greek, University of Leeds.

Please login or register to read this article.

Register to continue

Get a month's unlimited access to THE content online. Just register and complete your career summary.

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Most commented

Recent controversy over the future directions of both Stanford and Melbourne university presses have raised questions about the role of in-house publishing arms in a world of commercialisation, impact agendas, alternative facts – and ever-diminishing monograph sales. Anna McKie reports

3 October


Featured jobs

PhD Position, Department of Geoscience and Petroleum

Norwegian University Of Science & Technology -ntnu

Senior Lecturer in Law

University Of The West Of England (uwe)

Lecturer in Marketing

Edinburgh Napier University