Review by peers 'good for science'

Study shows current system works. Anna Fazackerley reports.

Academics should stop complaining about the pressures of the peer-review system and start working to educate the public about its importance, according to a report published this week.

A working party of scientists, convened by campaign group Sense about Science, spent more than a year examining the much-criticised peer-review system - the method of assessing whether research merits publication - that has been used for at least a century.

The group's final report, published on Thursday, concludes that scientists are bogged down by the responsibility to submit and review papers and that this had caused them to lose sight of the benefits of the system.

It says the biggest problem with peer review is that so few members of the public are aware of it.

Tracey Brown, director of Sense about Science, told The Times Higher : "You could say we live in a time when people have a hunger for science like never before. They are very open to finding some means by which they can filter the information they are given."

She added: "It took our group a while to stop worrying about their own hobbyhorses and think more broadly about the system."

The working party called for a culture change both in and outside academia, with researchers placed under pressure by the media, politicians and the public to explain how rigorous their work is.

Ms Brown said anyone interested in scientific findings should automatically ask tough questions such as "have these results been peer reviewed?" and "have they been published in a recognised scientific journal?"

The report acknowledges that peer review can never be infallible, because reviewers can always make mistakes and the system can never be a "fraud detector".

But it says reviewing is only the first stage and a paper that is published must go on to be retested and judged against other work in the field.

The report attacked scientists who choose to release research about new threats to health or the environment directly to the public in order to leapfrog the lengthy peer-review process.

It says that although waiting for a paper to be published could be frustrating, the potential costs of going it alone and promoting bad research are "enormous".

Ms Brown said: "If someone is presenting work that hasn't been peer reviewed, we should be asking why. What are they afraid of?"

She added: "The more stir a paper causes, the more likely it is that problems will be found quickly, as people will want to repeat it."

The working party warned that the public was often attracted to claims that had not been endorsed by experts, perhaps wanting to believe that "authentic" information was that which defied accepted ideas.

HOW TO GO PUBLIC

• Scientists should regularly draw attention to the fact that their work has been peer-reviewed when discussing it
• Academics should work with press officers to ensure their peer-reviewed work is reflected accurately in publicity, and press officers must understand the peer-review system
• Scientists should follow media coverage of their research and correct claims that deviate from the peer-reviewed work
• Conference documents and all promotional material should state whether work has been peer reviewed
• Science education bodies should produce educational resources about peer review for schools. In higher education, all courses covering risk assessment and the philosophy of science should include education about peer review.

SCARES

Non-reviewed research

• In 1998, self-employed researcher Roger Coghill released research straight to the media saying that the waves produced by mobile phones could damage the activity of lymphocytes in the body's immune system. Over the next five years his claims, though contradicted by most other evidence, were cited in 119 news publications in the UK alone. Most made no reference to the unofficial status of the research paper or to the fact that other research disagreed.
• Paul Shattock, a pharmacist who set up the autism research unit at Sunderland University, made headlines in 2002 with non-peer reviewed research claiming to have identified a group of children whose autism was caused by the MMR vaccine. No scientific journal accepted the research, but it fuelled the scare about the vaccine and was cited in 41 newspaper articles.
• In April 2002, research from Stockholm University suggested that people were at risk of cancer from consuming acrylamide in heated fatty foods. This sparked a world health scare, renewing concerns about the dangers to children of eating crisps. Later studies failed to repeat the findings.

Better the devil we know

"You can get inundated with requests for reviews. When you have all sorts of other things to do, you get tired of being asked to provide opinions that will, at best, be mixed," said John Winfield, a professor in the department of chemistry at Glasgow University.
Professor Winfield, who is a regional co-editor for a journal as well a regular peer reviewer for chemistry journals, admitted that peer review had become a favourite hobbyhorse for disgruntled academics.

However, he was adamant that it must not be replaced.

"We must keep it," he said. "I have no hesitation in defending it because I can't think of any system to replace it."

He added: "Peer review is part of the fabric of academic life. We've all grown up with it, and no one thinks it is perfect. But it's like democracy - it's the best system we have."