Can post-publication peer review endure?

The process may mean greater scrutiny but is it legally viable?

November 13, 2014

Source: Getty

On the lookout: critics have suggested that if commentators were made to reveal their identities then criticism may be fairer, more civil and less likely to result in legal action

The legal action launched by a US scientist who claims that anonymous comments questioning his science cost him a lucrative job offer has raised further questions about the potential for post-publication peer review to replace pre-publication review.

Academics’ gripes with prepublication review are well documented. When only two or three reviewers are asked to comment on a manuscript, the potential for acceptance decisions to be skewed by misunderstanding or bias is considerable. Proponents of post-publication review say scrutiny by the whole community increases the chances of errors being spotted and significance being accurately identified. Some even hope that journal publishing – and its vast associated expense – could be dispensed with if there is no longer a need for editors to organise peer review.

Advocates’ current darling is PubPeer, an “online journal club” set up in 2012 by an anonymous group of early career researchers that allows users to comment – anonymously or otherwise – on published papers. The site’s rapid rise to prominence was highlighted earlier this year by the role it played in exposing the two now-infamous and retracted Nature papers by Haruko Obokata, of Japan’s RIKEN Center for Developmental Biology, that claimed to detail a new way to create stem cells.

However, ever since the tendentiously named Science Fraud website – which listed doubts about published papers – was shut down in early 2013 under a barrage of libel challenges, doubts have lingered about whether post-publication peer review is legally viable.

Most pre-publication reviews are never made public, so even one that is highly critical is no threat to a researcher’s wider reputation. Post-publication review, by contrast, is necessarily open, so critical reviews are more likely to elicit a robust response – which, in an increasingly litigious society, could involve lawyers. As Nature recently admitted in an editorial, fear of legal challenge is one reason why it has “concluded that we cannot usually use retraction statements as a means of highlighting wrong-doing” – even when official investigations have reached misconduct verdicts.

Fears of “lawyered up” academics will be strengthened by the case of Fazlul Sarkar, a distinguished professor in cancer research at Wayne State University in Detroit. As first reported by Retraction Watch, he claims that anonymous comments posted on PubPeer this summer led to the withdrawal of a $350,000 (£220,000) a year job offer by the University of Mississippi.

Under US law, PubPeer is not liable for the remarks of its commenters, but Sarkar has subpoenaed it to demand the identity of those he believes to have libelled him – whom he intends to sue. The site responded by taking down the offending comments but said it will resist the subpoena and will cease in future to hold any information – such as an IP address – that might identify its anonymous contributors.

Behind the usernames

Some observers have suggested that the anonymity PubPeer offers is problematic because it offers impunity to those who slip past its moderators and unfairly shred others’ reputations. If commenters were required to reveal their identities, some argue, criticism would be fairer, more civil and less likely to come to a lawyer’s attention.

One proponent of that view is Philip Moriarty. Earlier this year, the professor of physics at the University of Nottingham had a heated exchange on PubPeer with defenders of Francesco Stellacci, Constellium professor at the École Polytechnique Fédérale de Lausanne in Switzerland, over whether Stellacci’s belief in the existence of “stripy nanoparticles” is due to basic errors in microscopy. Moriarty says he was not concerned that Stellacci – who has complained of a witch-hunt – might sue him, but he admits that attitude may just reflect his own “combination of naivety and stupidity”. But however robust discussions become, he believes critics should never retreat behind anonymity, and should always be compelled to identify themselves – at least to moderators. “All you need is one disgruntled postdoc or PhD student and anonymous commenting can be really damaging,” he says.

But, according to an emailed comment from PubPeer, its users have rarely abused their anonymity: “Abusive comments with no substance are both ineffective and easily spotted. Conversely, if a comment makes a valid point, the motivation for posting it is irrelevant. The worst we see is misguided comments made because of some misunderstanding – but, in that case, the authors or other users can explain, and it is probably beneficial for such explanations to be available to all.”

The email admits that anonymous comments are “often the most critical and may highlight features indicative of misconduct”. But it also insists they are important since “the biggest problems matter the most”.

Moriarty concedes that having to reveal their identities even to a moderator could put off vulnerable early career researchers, and he suspects that PubPeer’s popularity – in contrast to some previous experiments in post-publication review – is down to the possibility of anonymity. But he suggests that finding a way to make comments citeable – and, hence, count towards scientific prestige – might warm up some cold young feet.

However, Dave Fernig, a professor in the University of Liverpool’s department of biochemistry, observes that discussion forums that lack anonymity contain “a lot of hagiography” and rarely provide new insight into a paper: “If we lose anonymity, then many will fear to criticise. The consequence is that work that needs questioning will stand and we will have moved from science to faith,” he says.

The UK’s notoriously punitive libel law was recast last year to make it harder for claimants to sue. A defence was introduced for websites that take down offending comments promptly, and peer-reviewed papers were exempted from the scope of the law. However, discourse around papers remains unprotected and the cost of defending a libel action means Fernig’s call for “a good test case” to see where the limits lie seems unlikely to attract willing participants.

However, US libel law is weighed more in defendants’ favour. So while PubPeer accepts that further legal “attacks” are inevitable, it notes that, despite Sarkar’s actions, its contributors continue to be “active”, and it is “optimistic that the robust legal protections for free speech and anonymity in the US will enable sites like PubPeer to defend themselves and their users effectively”.

As for Moriarty, he is crossing his fingers that such confidence is not misplaced: “If you are publicly funded and you put your research into the public domain but no one can criticise you for it without facing legal proceedings, that seems to me to be a very badly damaged system.”

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Reader's comments (2)

A good thought-provoking article. I do believe strongly in open reviewing, having participated as a reviewer on the Educational Journal of Living Theories ( In this online publication submitters can post a paper for review. All review happens online and in public, and other people than the designated reviewer can also post publicly, and all are identified by name. In my experience, this has led to improved papers and often a dialogue between submitters and reviewers, where the latter have occasionally changed a critical comment when made aware of local nuances or practices whose relevance was not immediately obvious, or else the submitters have clarified in the paper why the comments were made. It seems to me that the critiques have been robust - not subject to the 'watering-down' that the article above suggests can happen when reviewers are identified. I have certainly learned a lot both from the overall process, and from responses from submitters, who can decide whether they will proceed with subsequent iterations to publication, or withdraw from the process. Few withdraw.
Can science endure with the present system of editors and reviewers who do not spot many errors?

Have your say

Log in or register to post comments

Featured Jobs

Organisation Design Specialist UNIVERSITY COLLEGE DUBLIN (UCD)
Dean of Health and Social Care STAFFORDSHIRE UNIVERSITY
Dean of Engineering and Creative Arts STAFFORDSHIRE UNIVERSITY
Dean of Law, Policing and Forensics STAFFORDSHIRE UNIVERSITY
Lecturer in Composition DURHAM UNIVERSITY

Most Commented

University of Oxford students walking on campus

University of Oxford snatches top spot from Caltech in this year’s World University Rankings as Asia’s rise continues

Home secretary says government will support 'best' universities

Man handing microphone to audience member

Academic attainment of disadvantaged students can be improved if they can decide how they are assessed, study claims

Italy's gold medallist

New measures to ensure universities are ‘not penalised’ for taking poorer students also outlined for next stage of TEF

Jean-Paul Sartre and Simone de Beauvoir meeting over coffee

Claims for genius require more than repeated assertion to make the case, says Martin Cohen