Citation pitfalls ahead

March 6, 2008

Charles Oppenheim (Letters, 21 February) says previous studies "have shown excellent correlations between research assessment exercise scores and citation studies and therefore indicate citation analysis would not greatly affect overall order of RAE score". But such correlations are consistent with individual outliers. A demonstrable correlation is no guarantee that citation metrics will reliably replicate rankings based on quality judgments for individual department. Since the point of the exercise is to allocate funding to individual departments, such discrepancies should not be glossed over.

There is a deeper problem. Quality of research is not the only factor influencing frequency of citation. In my discipline, some topics receive far more attention than others: for example, publications on Homer are more than 30 times more numerous than those on Heliodorus. So if you want to maximise citations, give up on Heliodorus - especially if you are engaged in research with a long-term payoff. Citations accumulate slowly in Classics, so you will need to focus efforts on provocative interventions in current debates to stand any chance of gaining citations that will influence the next funding round.

A funding regime that selectively rewards quick-return contributions on a narrow range of intensively discussed topics will drive high-quality, low-citation research to extinction. That would do wonders for the correlation between quality judgments and citation metrics, but the consequences for the health of the discipline would be horrendous.

Touting correlations without understanding the causal relations that underpin them is a perfect recipe for creating perverse incentives.

Malcolm Heath, Professor of Greek, University of Leeds.

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments

Most Commented

James Fryer illustration (27 July 2017)

It is not Luddism to be cautious about destroying an academic publishing industry that has served us well, says Marilyn Deegan

Jeffrey Beall, associate professor and librarian at the University of Colorado Denver

Creator of controversial predatory journals blacklist says some peers are failing to warn of dangers of disreputable publishers

Hand squeezing stress ball
Working 55 hours per week, the loss of research periods, slashed pensions, increased bureaucracy, tiny budgets and declining standards have finally forced Michael Edwards out
Kayaker and jet skiiers

Nazima Kadir’s social circle reveals a range of alternative careers for would-be scholars, and often with better rewards than academia

hole in ground

‘Drastic action’ required to fix multibillion-pound shortfall in Universities Superannuation Scheme, expert warns