Concern over tacit conflicts of interest in PLoS ONE peer reviews

Loughborough academic alleges ‘process problem’ with journal’s trust-based system

January 30, 2014

Source: Alamy

Too close? PLoS says its policy obliges editors to declare competing interests

The peer review system of the world’s largest journal could be subject to widespread abuse by authors, editors and reviewers who fail to declare conflicts of interest, an academic has claimed.

Jamie Timmons, professor of systems biology at Loughborough University, said he had contacted the California-based PLoS ONE eight months ago with several examples of papers written by colleagues or collaborators of the academic editor who oversaw their acceptance.

A spokesman for PLoS ONE, which published 31,000 articles last year, said academic editors are approached by an automated system matching their expertise to the manuscript in question. However, authors were also free to suggest their own editors, who were typically placed at or near the top of the list of editors to be approached.

Authors, editors and peer reviewers were all obliged to declare any competing interests and, where necessary, reviewers and editors had to “recuse themselves”. The spokesman admitted the system was “based on trust”, but said the journal polices compliance with its conflict of interest policy and “takes seriously” any breaches.

Regarding the papers Professor Timmons had highlighted, the journal had followed its policy of asking another member of its editorial board to reassess them. In the case of one paper, the original publication decision had been endorsed, but the academic editor in question – José Calbet, professor of exercise physiology at the University of Las Palmas de Gran Canaria in Spain – “is not handling any additional submissions as our investigations proceed”.

A comment posted by PLoS under the article explained this situation and similar comments had been posted on three additional papers that came to light during PLoS’ investigation, the journal’s spokesman said. Other papers were still being re-evaluated.

However, Professor Timmons said posting notes was “pointless” because very few people would read them. Nor would reassessing the original peer review reports be sufficient since a conflicted editor might have chosen “sympathetic” referees.

He also suggested that the size of PLoS ONE, which has about 5,000 editors and publishes everything judged to be scientifically sound, made it practically impossible to police conflicts of interest effectively. He suspected a more systematic search would uncover “hundreds or more” of examples of conflicts of interest.

“Given that it is certain that these editors and authors worked together in the past and present…it is certain PLoS ONE has a process problem with its peer-review system,” he said.

He called on PLoS to publicly acknowledge the problem and explain how it would investigate its extent. “As things stand, PLoS ONE can’t be considered a properly peer reviewed journal – not because others don’t have this problem but because you can prove PLoS ONE has [this problem] as it uniquely publishes the name of the editor on each article,” he said.

Professor Calbet said he had not realised that there was potential conflict of interest regarding the paper highlighted by Professor Timmons and had assumed that PLoS ONE’s editorial office would not have approached him if a potential conflict existed.

“But a potential conflict of interest does not necessarily mean wrong-doing,” he added. “The full process has been examined and…the editorial decision was considered fair and not biased.”

paul.jump@tsleducation.com

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (7)

After a 10 min inspection there were over a dozen such articles, in the area I was interested in, including editors editing for colleagues from the same academic department! I'd encourage every one too look at their area of interest and see how deep this problem with PLOS is. Its tax payers money that funds this journal through publication fees, and its about scientific integrity. Regarding Dr Calbet's claims, he worked with the co-authors for many years in Copenhagen (Pillegaard et al) and was con-currently publishing with them in PLOS and other journals. Shocking stuff. Jamie Timmons
The problem of abuse that Jamie Timmons flags is a problem that all journals battle with, not just PLoS One. All reputable journals have policies in place to ensure that handling Editors and potential peer reviewers declare any conflicts of interest before embarking on peer review. However, the whole system of peer review is reliant on trust; from trusting that authors are real, to trusting that data is not fabricated, to trusting that reviewers and Editors declare their conflicts of interest... PLoS One is also not unique in publishing the name of the handling Editor, on the peer-reviewed published article. E.g. the Frontiers journals do this. And one would have thought this goes some way to ensuring the system is more transparent, by allowing the reader to identify and consider any potential bias. Another approach to counter this problem is using a system of fully open peer review. BioMed Central operates open peer review on the medical titles in the BMC series (and has done for the past 10 years), and more recently biology titles too, for example, Biology Direct and GigaScience. This ‘openness’ is on two levels. The first is that authors will naturally see the reviewers' names; the second is that if the article is published, the reading public will also see who reviewed the article and how the authors responded. It makes the process transparent, makes the reviewers more accountable and gives credit. We’ve also found the quality of reviewer reports is higher under a system of open peer review. Biology Direct goes further and allows authors to select suitable reviewers from the journal's Editorial Board, in a fully open and transparent way making peer review truly collaborative. In this scenario, you could indeed have a close colleague openly handle a friends manuscript, but be empowered to choose the hottest critics to review the work openly without fear of accusations of bias. So yes, a potential conflict of interest does not necessarily mean wrong-doing. F1000R value openness in their post-publication peer review approach too. Elizabeth Moylan, Biology Editor, BioMed Central
As Elizabeth Moylan mentioned above, F1000Research (like BioMed Central) uses an open peer review model. All of our referees are named, and all referee reports are visible. Peer review is carried out post-publication and an article passes peer review when it has reached a certain threshold of referee approval. Because this puts a lot of power in the hands of the referees, we spend a lot of time checking for conflicts of interest. But you can imagine that there are rare instances in which editorial staff will not be able to spot an existing conflict of interest, for example when referee and author have only recently started collaborating and there is no evidence of this yet. In these situations, publishing referee names is a great system to catch any conflicts that slip through the cracks. First of all, it makes the referee think twice before failing to declare a conflict of interest, and second, others familiar with the field and ongoing collaborations are able to see exactly who said what. Doing this post-publication allows us to, for example, invite additional referees at a later time to add new voices to the discussion if a potential conflict comes to light, and readers can comment on a paper to share their opinion, even while the review process is still ongoing. It's true that readers are not going to do a systematic check for conflicts on every paper, but they do read papers in their field, written and reviewed by people they know, and they do notice when something is off. In our experience, we have found that readers will highlight such conflicts if they see them, and this in turn helps to keep our referees on their toes. Eva Amsen, Outreach Director F1000Research
A distinguishing feature of PLOS ONE is that the main criterion for acceptance of publication is whether the conclusions of a paper are supported by the data presented, the data themselves obtained by appropriate methodology and subjected to a clear (rational?) analysis. There are no requirements on significance, priority etc. (except, of course, a requirement that the same data do not appear elsewhere or if repetitions are present that they make sense). Combining the above with openness, which I see is not being disputed here, generates a reasonably good platform for scientific publication, which may be one reason for the Journal's relative success. The examples above simply prove how openness can be used to show potential connections. Whether there is anything inherently problematic about such connections can be left to the judgment of readers (again examples above). The quality of the peer review process, in the form it takes within PLOS ONE, will depend on the individual players. I recall the times when members of the USA National Academy of Sciences could publish their research in PNAS and the mixed reaction of getting a paper published in that prestigious venue that ensued as a result. I think as a community we face a difficult problem. On the one hand everyone benefits if the published literature is of high quality - having numerous studies whose conclusions cannot be substantiated by the experiments performed generates a headache to the student. On the other hand everyone suffers if high quality science remains untold because of too much restriction. At the top of the problem is the strong cultural acceptance of the "publish or perish" rule. I would be interested to know about alternative ways of assessment for individual researchers, where the huge heterogeneity of character can be better accommodated. We are bleeding talent away by only selecting the strong and fast.
Re Fanis's last comment, Kent Anderson of the Scholarly Kitchen wrote a piece earlier this week about Plos One and the publish or perish culture, which people might want to look at: http://scholarlykitchen.sspnet.org/2014/01/29/can-mega-journals-maintain-boundaries-when-they-and-their-customers-both-embrace-publish-or-perish/
The problem is endemic in the editing of psychology journals, that is why I have refused to join the editorial board of Journal of Consulting and Clinical Psychology, when I was invited. The APA flagship Journal American Psychologist is even worse. The problem is just more detectable at PLOS One because of the laudable process of indicating who was the academic editor for a published paper. It should also be pointed out that PLOS One has over 4000 academic editors, and this problem, though it should be taken seriously, is limited to a very small minority. By way of disclosure, I am a (unpaid) academic editor at PLOS One and like my fellow academic editors, I am quite committed to the highest quality open access that PLOS One represents. But my comments here represent my views, and should not be construed as representing PLOS One.
In an earlier comment on this article, Dr Jamie Timmons raised concerns about a competing interest by Dr Jose Calbet in his handling of several submissions to PLOS ONE authored by his collaborators. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision making, or publication of research or non-research articles submitted to one of the journals (http://www.plosone.org/static/competing.action). The PLOS policy on competing interests lists a five-year period as a guide for situations arising that may constitute a competing interest. Dr Timmons listed a number of PLOS ONE articles authored or edited by Dr Calbet. PLOS ONE takes potential competing interests seriously and we have followed up on the concerns raised. The following publication was authored by researchers with whom Dr Calbet collaborated in the five years that preceded the submission to the journal: Skeletal Muscle PGC-1? Is Required for Maintaining an Acute LPS-Induced TNF? Response (10.1371/journal.pone.0032222) PLOS ONE does consider this to constitute a competing interest and as a result the journal has completed a re-evaluation of the peer review process of the article by an independent editorial board member. This re-assessment has established that the peer review process and editorial decision for publication were adequate and in line with the journal’s criteria. A comment has been posted on the article to reflect this finding. With regard to the other publications listed in Dr Timmons’ post, several were published in journals other than PLOS ONE and several others are articles authored by Dr Calbet that were handled by Academic Editors with whom Dr Calbet has not co-authored publications, or where the co-publications occurred beyond the five-year period noted above. As a result, we consider that the collaboration on those publications falls beyond the stipulations outlined in the competing interests policy. Damian Pattinson, Editorial Director, PLOS ONE

Have your say

Log in or register to post comments

Featured Jobs

Head of Visual Arts UNIVERSITY OF HERTFORDSHIRE
Research Officer - Big Data for Better Outcomes LONDON SCHOOL OF ECONOMICS & POLITICAL SCIENCE LSE
Lecturer in Oral Microbiology UNIVERSITY OF BRISTOL

Most Commented

question marks PhD study

Selecting the right doctorate is crucial for success. Robert MacIntosh and Kevin O'Gorman share top 10 tips on how to pick a PhD

India, UK, flag

Sir Keith Burnett reflects on what he learned about international students while in India with the UK prime minister

Pencil lying on open diary

Requesting a log of daily activity means that trust between the institution and the scholar has broken down, says Toby Miller

Application for graduate job
Universities producing the most employable graduates have been ranked by companies around the world in the Global University Employability Ranking 2016
Construction workers erecting barriers

Directly linking non-EU recruitment to award levels in teaching assessment has also been under consideration, sources suggest