Journal to rank peer reviewers to improve response times

Idea is one of several ways to speed the peer review process, although some think it will annoy academics

September 7, 2016
Egg timer and clock showing deadlines

A behavioural economics journal is trialling a novel and controversial way of making sure peer reviewers submit their reviews on time – releasing a ranking of how quickly they respond.  

Advocates of the pilot scheme by the Journal of Behavioral and Experimental Economics argue that it will reward speedy reviewers, but critics say that it will only make academics more “fed up” of offering “free labour” to journals.

Journals, which generally do not pay reviewers or make their reviews public, normally have no carrots or sticks to incentivise a quick turnaround, and rely on academics’ goodwill.

However, with the peer review system creaking at the seams because of the ever-increasing number of articles requiring review, journals are experimenting with new ways to speed the process along, such as offering payment.  

The journal will release a ranking of reviewer speeds on its website, calculated by the time between an invitation to review and submission. However, the slowest 20 per cent will not be revealed, and the actual number of days taken will also be kept private.

The aim is to “encourage efficient and timely reviewing and to recognize the appreciation for the important work of reviewers”, according to an email sent out to reviewers at the end of last month.

But the idea has attracted criticism, with some academics saying the idea will only antagonise reviewers. Lenny Teytelman, a co-founder of Protocols, a repository for life sciences experimental methods, told Times Higher Education that academics were “so fed up” of offering “free labour” that “saying we’re going to rank you, it just exacerbates the problem”.

The fact that the journal is owned by the publishing giant Elsevier, which has been criticised for what some see as excessive profit margins, has stoked criticism further. “In this case, for a corporation which has really good profit margins, the optics aren’t good,” he added. However, he said that he thought the technique would be effective, even if unfair.

Jim Woodgett, director of research at the Lunenfeld-Tanenbaum Research Institute in Toronto, said that he found the idea “ridiculous”.

“While authors clearly would love to have rapid assessment of their papers, they also want their work evaluated by the most appropriate experts and would, I am sure, prioritise that over whether the review shaved off a few days,” he said.

He also asked: “How is this an incentive? Publishers and editors seem to think that scientists only review for them.”

A spokesman for Elsevier said that the idea was “only a pilot with one journal, alongside many pilots we have to innovate peer review. It’s too early to tell if the journal will continue with this approach in the long term, but reviewer feedback will be monitored over the coming months.”

He pointed out that there had been other experiments with making reviewer speeds “more transparent”. In 2014, the Journal of Public Economics, also owned by Elsevier, released the results of a study that looked at different approaches to speeding up peer review.

Telling reviewers that their turnaround times would be posted publicly was “less effective” than offering $100 (£75) to hit a deadline, but was “still surprisingly successful in reducing review times, particularly with tenured professors, who were less sensitive to cash and deadlines”, it found.

Many journals have a “big problem” in finding peer reviewers, said Dr Teytelman, as “there’s been such an explosion in the number of articles published”. But he said that the best solution was to give reviewers public credit for their work, not rank their speed.

He pointed to one such initiative, Publons, a tool that allows academics to verify how much reviewing they have done so that they can use their record when applying for promotions or funding.

david.matthews@tesglobal.com

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

POSTSCRIPT:

Print headline: Journal to rank speed of peer reviewers to improve response times

Have your say

Log in or register to post comments

Most Commented

Lady Margaret Hall, Oxford will host a homeopathy conference next month

Charity says Lady Margaret Hall, Oxford is ‘naive’ to hire out its premises for event

Laurel and Hardy sawing a plank of wood

Working with other academics can be tricky so follow some key rules, say Kevin O'Gorman and Robert MacIntosh

Woman pulling blind down over an eye
Liz Morrish reflects on why she chose to tackle the failings of the neoliberal academy from the outside
White cliffs of Dover

From Australia to Singapore, David Matthews and John Elmes weigh the pros and cons of likely destinations

Michael Parkin illustration (9 March 2017)

Cramming study into the shortest possible time will impoverish the student experience and drive an even greater wedge between research-enabled permanent staff and the growing underclass of flexible teaching staff, says Tom Cutterham