Journal to rank peer reviewers to improve response times

Idea is one of several ways to speed the peer review process, although some think it will annoy academics

September 7, 2016
Egg timer and clock showing deadlines

A behavioural economics journal is trialling a novel and controversial way of making sure peer reviewers submit their reviews on time – releasing a ranking of how quickly they respond.  

Advocates of the pilot scheme by the Journal of Behavioral and Experimental Economics argue that it will reward speedy reviewers, but critics say that it will only make academics more “fed up” of offering “free labour” to journals.

Journals, which generally do not pay reviewers or make their reviews public, normally have no carrots or sticks to incentivise a quick turnaround, and rely on academics’ goodwill.

However, with the peer review system creaking at the seams because of the ever-increasing number of articles requiring review, journals are experimenting with new ways to speed the process along, such as offering payment.  

The journal will release a ranking of reviewer speeds on its website, calculated by the time between an invitation to review and submission. However, the slowest 20 per cent will not be revealed, and the actual number of days taken will also be kept private.

The aim is to “encourage efficient and timely reviewing and to recognize the appreciation for the important work of reviewers”, according to an email sent out to reviewers at the end of last month.

But the idea has attracted criticism, with some academics saying the idea will only antagonise reviewers. Lenny Teytelman, a co-founder of Protocols, a repository for life sciences experimental methods, told Times Higher Education that academics were “so fed up” of offering “free labour” that “saying we’re going to rank you, it just exacerbates the problem”.

The fact that the journal is owned by the publishing giant Elsevier, which has been criticised for what some see as excessive profit margins, has stoked criticism further. “In this case, for a corporation which has really good profit margins, the optics aren’t good,” he added. However, he said that he thought the technique would be effective, even if unfair.

Jim Woodgett, director of research at the Lunenfeld-Tanenbaum Research Institute in Toronto, said that he found the idea “ridiculous”.

“While authors clearly would love to have rapid assessment of their papers, they also want their work evaluated by the most appropriate experts and would, I am sure, prioritise that over whether the review shaved off a few days,” he said.

He also asked: “How is this an incentive? Publishers and editors seem to think that scientists only review for them.”

A spokesman for Elsevier said that the idea was “only a pilot with one journal, alongside many pilots we have to innovate peer review. It’s too early to tell if the journal will continue with this approach in the long term, but reviewer feedback will be monitored over the coming months.”

He pointed out that there had been other experiments with making reviewer speeds “more transparent”. In 2014, the Journal of Public Economics, also owned by Elsevier, released the results of a study that looked at different approaches to speeding up peer review.

Telling reviewers that their turnaround times would be posted publicly was “less effective” than offering $100 (£75) to hit a deadline, but was “still surprisingly successful in reducing review times, particularly with tenured professors, who were less sensitive to cash and deadlines”, it found.

Many journals have a “big problem” in finding peer reviewers, said Dr Teytelman, as “there’s been such an explosion in the number of articles published”. But he said that the best solution was to give reviewers public credit for their work, not rank their speed.

He pointed to one such initiative, Publons, a tool that allows academics to verify how much reviewing they have done so that they can use their record when applying for promotions or funding.

david.matthews@tesglobal.com

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

POSTSCRIPT:

Print headline: Journal to rank speed of peer reviewers to improve response times

Have your say

Log in or register to post comments

Featured Jobs

Deputy Chief Examiner for Spanish ab initio INTERNATIONAL BACCALAUREATE
Deputy Chief Examiner for Music INTERNATIONAL BACCALAUREATE
Deputy Chief Examiner for Visual Arts INTERNATIONAL BACCALAUREATE
Deputy Chief Examiner for Mathematics HL INTERNATIONAL BACCALAUREATE

Most Commented

James Fryer illustration (8 September 2016)

Some lecturers will rightly encourage forms of student interaction that are impossible for those covering their faces, Eric Heinze argues

University of Oxford students walking on campus

University of Oxford snatches top spot from Caltech in this year’s World University Rankings as Asia’s rise continues

Handwritten essay on table

Universities must pay more attention to the difficulties faced by students, says Daniel Dennehy

Theresa May entering 10 Downing Street, London

The prospect of new grammar schools on the horizon raises big questions for HE, writes Nick Hillman

Nosey man outside window

Head of UK admissions service Mary Curnock Cook addresses concerns that universities might ‘not hear a word’ from applicants