Amgen launches new platform to help fix scientific ‘reproducibility crisis’

Drug company hopes to provide a new home for studies that try to replicate previous experimental results

February 4, 2016
Ivan Timofeenko, Newton Park, Mirror Box
Source: Reuters
Same again? It’s not so easy to replicate experimental results in science

A major pharmaceutical company is launching a new platform for the publication of experiments that attempt to replicate previous research in a bid to tackle the “reproducibility crisis” in science.

In 2012, Amgen alarmed the scientific world by revealing that it had been able to reproduce the results of only six out of 53 “landmark” cancer studies. This confirmed similar, worrying findings from German drug company Bayer released the previous year.

Launched on 4 February, the Preclinical Reproducibility and Robustness channel is on the F1000Research publishing platform, which is open access and invites peer review after publication.

Bruce Alberts, a prominent biochemist at the University of California, San Francisco and former editor-in-chief of Science, said it would be a home for replication attempts that were often shunned by journals.

“One of the problems with science is that there’s a bias against publishing [studies] to reproduce” results, said Dr Alberts, who is helping to promote the channel.

So far, in-house attempts to reproduce results by drug companies had not generally been made public, he explained. The first papers published on the channel are three replication studies by California-based Amgen – which all fail to confirm the results of previous experiments.

The inability to reproduce results has been blamed on a number of factors, from statistically underpowered studies to the post-hoc manipulation of data.

Some have criticised high-profile journals like Science for exacerbating the reproducibility problem by claiming they demand novel, positive and media-friendly results, which can create pressures to come up with a certain experimental outcome.

Asked about the role of his former magazine, Dr Alberts said that “all journals only want positive results”.

“What you could say about these high-profile journals is that they want a finding that has a big impact”, something that could encourage dishonest scientists to hunt for a particular result, he said.

To deal with this, journals like Science needed a “very, very high standard for [peer] review”.

During his period at Science, Dr Alberts said he had resisted telling scientists that they would be published only if they completed further experiments and found a similar outcome, a practice he said was “very distorting” to the scientific process (since it could create an incentive to get the same result by whatever means).

But the pressures journals put on researchers was only a “small part of the problem”, he said. More important was setting better rules for proper scientific procedure.  

However, science should not aim for a 99 per cent reproducibility rate, he added. “You don’t want every paper to be perfect because that would delay publication” and stifle scientists’ ability to put out new ideas, he argued.

Dr Alberts said that unless the problem was addressed, there was “of course” a risk that scientists could lose public trust in the same way journalists and politicians had done.  

Like free market capitalism, “with no rules and oversight [science] becomes very distorted” and up until now there had not been enough efforts to fix problems. 

Marcia McNutt, editor-in-chief of the Science family of journals, said its track record showed it had been "a leader in raising standards for reproducibility and publishing reproducibility studies, including those with negative findings". She gave the example of a recent paper it published probing the replicability of 100 landmark papers in psychology.

"We absolutely want everything we publish to be interesting, groundbreaking, and thought-provoking, but we never demand positive results. Disproving a reigning paradigm might be just as interesting as breaking new ground,” she added.

Please login or register to read this article

Register to continue

Get a month's unlimited access to THE content online. Just register and complete your career summary.

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Related articles


Print headline: Amgen set to tackle reproducibility crisis

Have your say

Log in or register to post comments