Citation database accuracy queried

Web of Science's owner defends procedures after academic complains of errors, Zoe Corbyn writes

November 6, 2008

It contains more than 600 million research paper citations and processes nearly 40 million references each year, so perhaps it is not surprising that the world's largest citation database is prone to the odd error.

But after discovering a bizarre anomaly, one academic claimed this week to have "lost faith" in the Web of Science (WoS) database - which is set to help determine the allocation of more than £1 billion a year in research funding after this year's final research assessment exercise.

Richard Hull, a senior lecturer in management at Newcastle University, selected a paper entitled "Job quality and part-time work in the retail industry", published in the International Journal of Human Resource Management in March 2008, to teach students how to use the WoS. But he became perplexed when the WoS said this paper cited a 1997 article on the genetic engineering of pigs in the journal Biodrugs.

He said that the "false" citation for the journal and paper suggested that trust in the validity of the WoS was "very misplaced".

It is not the first time Dr Hull has highlighted such errors. Last September he found several mistakes that were reported in Times Higher Education. He said he would have expected database owner Thomson Reuters to tighten up procedures.

Marie McVeigh, senior manager of bibliographic policy at Thomson Reuters, said that the company used both automated processes and people to ensure that data were correct. While any error was regrettable, the frequency of occurrence was "extremely small", she said. In the case in question, the company has now identified the underlying cause of ambiguity and corrected it.

"(The database) remains the highest quality and most comprehensive and trusted citation database in the world," she said.

Dr Hull said his discoveries raised wider questions about what such anomalies might mean when databases are used to judge the quality of academics' work in the research excellence framework (REF).

The WoS and the Elsevier-owned Scopus database are being considered by the Higher Education Funding Council for England (Hefce) to count academics' citations, as a proxy for judging research quality under the REF.

Dr Hull mooted that Hefce might wish to establish an independent authority to "validate and certify" all data used for bibliometrics.

A spokesman for Hefce said citation data would need to be "sufficiently robust" for the REF and, depending on what a current pilot discovered, it would take measures to ensure it was.

Jonathan Adams, director of the data analysis firm Evidence Ltd, said Thomson was "incredibly good" at correcting faults.

Please login or register to read this article

Register to continue

Get a month's unlimited access to THE content online. Just register and complete your career summary.

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Have your say

Log in or register to post comments