Quality of an academic paper counts for more than the prestige of the publication in which it appears, says a study that raises questions about RAE and metrics criteria. Jessica Shepherd reports
It is better for an academic paper to be one of the best in a medium-ranking journal than to be one of the weakest articles in a world-leading title, a study has revealed.
An examination of six journals over the past quarter of a century shows that academics who published in the least prestigious publications were often cited more by other academics than those with papers in the most prestigious journals.
Although the study is based on a relatively small number of publications in one field, the author said the findings challenged the received wisdom governing the academic career ladder: that publishing in big-name journals is best the way to get noticed by academic colleagues and secure promotion.
It also raises questions about the priority accorded to articles in top journals by referees in the research assessment exercise.
This might be further emphasised in a metrics-based system for allocating research grants.
Andrew Oswald, professor of economics at Warwick University, discovered two academics who had published in the most respected economic journal - American Economic Review - but had not had their papers cited once. Two others had been cited three times.
Four academics published in the less prestigious Oxford Bulletin of Economics and Statistics , however, had been cited between five and 50 times. Both journals came out in winter 1981.
Professor Oswald said: "This paper finds it is far better to publish the best article in an issue of a medium-quality journal, such as the Oxford Bulletin of Economics and Statistics , than to publish the worst article in an issue of a top journal such as the American Economic Review ."
He believes young researchers in particular do not realise this and are "taken in by an obsession with the names of journals".
He said: "There is a tremendous pressure in my discipline to publish in certain high-prestige journals.
"What I have found is that often the high-prestige journals publish a few of the most important papers, but it is easy to forget that they also publish some really quite bad papers too."
One of the papers that appeared in the "top" title had 401 citations, but another had none. One of the papers that appeared in the least prestigious publication, meanwhile, had 50 citations.
Professor Oswald added: "The RAE meant many academics stopped applying common sense and started applying snobbery about journals. In fact, the most important thing is not which journal a paper appears in but why this paper is interesting."
Professor Oswald said the metrics system, which may replace the RAE, could make things worse. Under such a system, grants could be allocated according to an academic's citations or volume of research papers.
He said: "This makes me doubtful of mindless metrics, though some role for metrics might be OK."
The study compared the number of citations over 25 years of 85 papers in six journals; the American Economic Review , E conometrica, Journal of Public Economics, Economic Journal, Journal of Industrial Economics and Oxford Bulletin of Economics and Statistics .
The data were provided by the Social Science Citation Index last month.
Pierre Regibeau, managing editor of the Journal of Industrial Economics and reader in economics at Essex University, said: "I think Professor Oswald's findings are correct and widely suspected in academe. The review process of a paper is random and therefore there are going to be mistakes. Some papers are wrongly rejected and others are wrongly accepted.
"The problem comes in particular when academics' promotion prospects are based on which journals they are published in. Ideally, an academic's papers should be read and their citations counted rather than being judged on the title of a journal."
Professor Oswald added: "These findings appear to be true rather generally across fields. Journal labels are not terribly reliable."