Nips, tucks and other deceptions

Illicit cosmetic surgery can transform a shoddy scientific paper - and leave referees feeling decidedly ill at ease, says Tim Birkhead

We live in a culture of seeing what we can get away with, whether it be speeding or parking on double yellow lines. The phenomenon will be familiar to anyone who has dealt with undergraduate assignments.

But it also occurs among PhD students, postdoctorates and professors.

A colleague was once asked by a journal editor to shorten a paper. He was incensed but later told me with glee how he solved the problem: he reformatted the paper, changed the font size, but hadn't removed a word. The editor didn't notice and accepted the paper - an innocent point scored against a stodgy editor that hardly constitutes scientific misconduct.

But how about cases that involve the cosmetic concealment of fundamental methodological flaws in scientific papers? This kind of getting away with it is widespread and detected only if the same referee sees several versions of the work. I recently read a paper reporting a novel result in a prestigious scientific journal. The paper seemed sound and I added it to my EndNote file.

A week later I was asked by the same journal to review a follow-up paper by the same authors. To my surprise the submitted manuscript was a disaster; it was riddled with poor methodology, none of the observations was made blind; there was pseudoreplication and the statistics were inappropriate. I sent a detailed report and recommended rejection. I didn't see the comments of the other referees, but the editor informed me that he had rejected the paper, so I assume they also felt it was weak. I thought about the disparity between the published paper and the submitted manuscript. When researchers submit a manuscript to a journal they may think they've sent it out in its Sunday best. But reviewers can spot cheap fabric and shoddy workmanship a mile away. The published version, of course, is the product of a makeover - a process that includes referees' and editors'

"what-to-wear" suggestions for improving the manuscript. This is part of the normal procedure of science. But often, and I suspect in this case, the improvement of a manuscript involves illicit cosmetic surgery.

Here's what happens. The paper goes to referees who ask: was the study done blind? "Yes," the authors answer, because if they say "What do you mean?"

or "No" the paper will be bounced. Such deception is hard to detect, but if the same referee gets a new version of the same paper from another journal, it is often frighteningly obvious. And that is what happened here. Just two weeks after the original paper had been rejected, I was asked by another journal to review a paper with an identical title. My heart sank. I told the editor I had already seen it and asked if they still wanted me to review it. Of course, they said. The manuscript was dramatically improved but so brief (the new journal had a tight word limit) that it consisted almost entirely of results and discussion - any shortcomings of the methodology were conveniently removed.

Now I have an ethical dilemma. Had I seen this most recent version and not seen the earlier ones, I would have pronounced it an elegant and interesting piece of work. Having seen the sequence of events unfold through successive submissions, it seems obvious that dubious methods had been used to enhance the manuscript's quality.

Should I tell the editor my full story? Is that ethical? Should I pretend I never saw the earlier version and say the manuscript is acceptable? Is that ethical? Answers on a postcard, please. No, don't bother.

Tim Birkhead is professor of behavioural ecology at Sheffield University.