Journals must stop blocking critical comments

Allowing faulty papers to go unchallenged damages integrity and threatens dangerous real-world consequences, says Peter Bowbrick

May 27, 2021
Illustration of person cutting content from speech bubble, representing demise of critical comments in journals
Source: Getty

Critical comments are of fundamental importance to academic publishing. They do more than peer reviewers do to keep academics from knowingly submitting bad or faked research. And they can be the first step in shifting scientific paradigms.

When I started writing critical comments in 1973, there was a presumption that editors would publish them if possible. They would check them, but people who write comments face a reply from the author, so they check their own work more carefully than any referee or editor ever would. Besides, errors are usually obvious once they have been pointed out.

Comments continue to be read and cited for years. In the week of writing, for instance, more than 200 researchers and students from 12 countries have read 10 of my comments and refutations; seven of these were written more than 20 years ago.

But 15 years ago, the climate in publishing began to change. Some American journals decided that critical comments showed up the fallibility of their vetting process, which could affect the commercial value of their brands. Instead of accepting that all refereeing is fallible, they started running comments past two referees, as well as the editor. Inevitably one of the three would object to something trivial, and this could then be used as an excuse to reject the comment.

A few years ago, I submitted a comment to a journal showing that a paper it had published broke all the rules for surveys. However, one referee disagreed with my statement that it was improper to use members of one organisation to interview members of another when comparing the two organisations (which is just one of the rules). The comment was suppressed, so I put it up on ResearchGate and Academia.edu instead. Now lecturers ask their students to read it before conducting a survey.

More recently, a UK journal developed a cast iron defence against criticisms. When I objected that one of its papers was littered with errors of commission and omission, my comment was considered by the entire editorial board – of 49 people, according to the journal’s website. This level of pre-publication scrutiny is unprecedented.

The 49 – including the editor – found no fault with my economics or facts, but would not even consider publishing my comment because, I was told, the editorial board “expects all comments and rejoinders to be worded in a collegial and constructive manner, tone and style. We feel that yours does fall rather below our usual conventions in this regard.” I did not make any ad hominem attacks or snide remarks – unlike the author of the paper I was criticising – so I asked for clarification. But all I received was a reiteration that I must rewrite my comment “in the sort of tone and style that is normal for contributions to this journal”.

No rational writer would comply with such a hostile demand. I am asked to rewrite my comment to meet secret criteria about toning down criticisms. And if by some miracle I meet these criteria without the criticisms vanishing, the editorial board will send my comment to two referees. Given that the journal’s referees accepted the appalling paper in the first place, how much faith can I have in them?

Even if they pass my comment, the editor may still reject it. Research shows that a significant proportion of editors in the sciences overrule the referees or rewrite their comments. And if the author makes a reply, however wrong, meaningless, or ad hominem, I have no confidence that I will be allowed to write a rejoinder.

I surmise that the journal’s objection is actually to my conclusions, rather than the way I present them; this is not the first time I have seen the words “collegial” and “constructive” used as a demand for omerta.

The journal says that “the priority of the Editorial Board is the journal”. Clearly, we have totally different moral stances. The paper I object to is on the economics of famine. Bad economics kills people – and I am squeamish about killing people. I do not see any moral justification for the continued existence of a journal that refuses to correct falsehoods in any field, but particularly this one. Less seriously, the journal is arguably committing an offence under the UK’s Consumer Protection from Unfair Trading Regulations 2008 by continuing to sell a paper it knows to be full of falsehoods.

It is still possible to submit refutations of entire research programmes to a range of journals. In practice, though, such refutations develop out of a few critical comments on specific papers – and these can only be submitted to the journal that published the paper in the first place.

This means that to avoid the risk of wasting their time, comment writers these days would be wise to seek assurances from the journal about publication before they even begin. The implications for academic integrity – not to mention people’s willingness to attempt to set the record straight in the first place – are obvious.

Peter Bowbrick has retired after a career in economic research and consultancy.

POSTSCRIPT:

Print headline: Journals must welcome critical comments, not set out to block them

Please Login or Register to read this article.

Register to continue

Get a month's unlimited access to THE content online. Just register and complete your career summary.

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Related articles

Reader's comments (3)

The author may be interested to hear about the website PubPeer, which provides a venue for post-publication peer review of the kind he refers to: https://pubpeer.com/ It's quite widely used by scientists, but perhaps not so familiar to those in social sciences.
My own experience of seeking an important correction. In 2015 was published the "Rising-falling mercury pollution causing the rising-falling IQ of the Lynn-Flynn effect, as predicted by the antiinnatia theory of autism and IQ" (1), in the peer-reviewed PsycInfo-indexed Personality and Individual Differences. Three years later was published "Flynn effect and its reversal are both environmentally caused", by Bernt Bratsberg and Ole Rogeberg, in the "prestigious" PNAS (Proceedings of the National Academy of Sciences) (2). Their paper failed to mention that there had been published that other paper three years earlier which had specified the exact "environmental factor" (which the unfaulted autism theory had correctly predicted against all expert assumptions). I wrote a commentary simply pointing this out. The editor claimed it was not an important enough point to merit mentioning. Really? In that case one has to wonder why was B&R's paper worth publishing when it failed to discover anything new and instead just tended to hide the fact of existence the mercury pollution paper. What's the point of slaving away at getting a paper published in a peer-reviewed PsycInfo-indexed journal when these other authors are given licence to just pretend it hasn't existed anyway? 1. https://www.researchgate.net/publication/273789709_Rising-falling_mercury_pollution_causing_the_rising-falling_IQ_of_the_Lynn-Flynn_effect_as_predicted_by_the_antiinnatia_theory_of_autism_and_IQ 2. https://www.pnas.org/content/115/26/6674
Science is becoming a religion rather than the pursuit of objective truth through triangulation and falsification...

Sponsored