'Publish or perish' culture distorting research results

Study finds link between academic productivity and positive outcomes. John Morgan reports

April 22, 2010

A "publish or perish" culture in which scientific careers rely on the volume of citations is distorting the results of research, a new study suggests.

An analysis by Daniele Fanelli, of the University of Edinburgh, finds that researchers report more "positive" results for their experiments in US states where academics publish more frequently.

He cites earlier research suggesting that papers are more likely to be accepted by journals if they report positive results that support an experimental hypothesis, producing a bias against "negative" results.

Dr Fanelli, Marie Curie research fellow at Edinburgh's Institute for the Study of Science, Technology and Innovation, sets his study against a context of "growing competition for research funding and academic positions, which, combined with an increasing use of bibliometric parameters to evaluate careers ... pressures scientists into producing 'publishable' results".

Some subject panels in the forthcoming research excellence framework, which will be used to distribute around £1.5 billion in annual quality-related research funding in England, will use citation data in their judgements of academics' outputs.

Dr Fanelli's study, published this week in the open-access journal PLoS ONE, states: "In a random sample of 1,316 papers that declared to have 'tested a hypothesis' in all disciplines, outcomes could be significantly predicted by knowing the addresses of the corresponding authors: those based in US states where researchers publish more papers per capita were significantly more likely to report positive results, independently of their discipline, methodology and research expenditure."

The results "support the hypothesis that competitive academic environments increase not only the productivity of researchers, but also their bias against 'negative' results", it adds.

Dr Fanelli titled the paper, "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data".

He told Times Higher Education: "There is quite a longstanding discussion about whether this growing culture of 'publish or perish' in academia is actually distorting the scientific process itself ... My study is the first to try to verify directly this effect in scientific literature across all fields."

Using data from the US National Science Foundation, the proportion of positive results in a random sample of papers published between 2000 and 2007 was set against a measure of "academic productivity" - the number of articles published per doctorate holder in academia in each US state - while "controlling for the effects of per-capita research expenditure".

Positive results accounted for less than half of the total in Nevada, North Dakota and Mississippi. However, states such as Michigan, Ohio, District of Columbia and Nebraska reported positive results for between 95 and 100 per cent.

The paper argues that negative results most likely "either went completely unpublished or were somehow turned into positive through selective reporting, post-hoc reinterpretation, and alteration of methods, analyses and data".

But does a link between citation rates and positive results necessarily mean the former caused the latter?

Dr Fanelli said: "This is not conclusive evidence. I would call it the starting point in this direction."

john.morgan@tsleducation.com.

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Most Commented

Lady Margaret Hall, Oxford will host a homeopathy conference next month

Charity says Lady Margaret Hall, Oxford is ‘naive’ to hire out its premises for event

women leapfrog. Vintage

Robert MacIntosh and Kevin O’Gorman offer advice on climbing the career ladder

Woman pulling blind down over an eye
Liz Morrish reflects on why she chose to tackle the failings of the neoliberal academy from the outside
White cliffs of Dover

From Australia to Singapore, David Matthews and John Elmes weigh the pros and cons of likely destinations

Michael Parkin illustration (9 March 2017)

Cramming study into the shortest possible time will impoverish the student experience and drive an even greater wedge between research-enabled permanent staff and the growing underclass of flexible teaching staff, says Tom Cutterham