Robot-written reviews fool academics

Peer reviews created by self-generated text machines are the latest threat to scientific integrity

September 20, 2016
Attacked by robot
Source: Rex
The machine age: although computer-generated reviews ‘cannot possibly deceive any rigorous editorial procedure’, they could ‘find a role in questionable scenarios and magnify the scale of scholarly frauds’, the paper finds

Soulless computer algorithms are already churning out weather bulletins, sports reports, rap lyrics and even passable Chinese poetry.

But it seems machines have now taken another step towards replacing human enterprise by generating their own reviews of serious academic journal papers that are able to impress even experienced academics.

Using automatic text generation software, computer scientists at Italy’s University of Trieste created a series of fake peer reviews of genuine journal papers and asked academics of different levels of seniority to say whether they agreed with their recommendations to accept for publication or not.

In a quarter of cases, academics said they agreed with the fake review’s conclusions, even though they were entirely made up of computer-generated gobbledegook – or, rather, sentences picked at random from a selection of peer reviews taken from subjects as diverse as brain science, ecology and ornithology.

“Sentences like ‘it would be good if you can also talk about the importance of establishing some good shared benchmarks’ or ‘it would be useful to identify key assumptions in the modelling' are probably well suited to almost any review,” explained Eric Medvet, assistant professor at Trieste’s department of engineering and architecture, who conducted the experiment with colleagues at his university’s Machine Learning Lab.

“If, by chance, a generated review combines sentences which are not too specific, but credible, the review itself may appear as written by a real, human reviewer even to the eyes of an experienced reader,” added Dr Medvet, whose paper, "Your Paper has been Accepted, Rejected, or Whatever: Automatic Generation of Scientific Paper Reviews", was published in the journal Lecture Notes in Computer Science last month.

Mixing the fake reviews with real reviews was also likely to distort decisions made by academics by making weak papers appear far stronger thanks to a series of glowing reviews, the paper found.

The research team was able to influence the peer review process in one in four cases by throwing fake reviews into the mix, it said.

“This [may be] the situation [faced by a] real conference programme chair…who has to take decisions about all the submissions at his or her conference,” Dr Medvet told Times Higher Education.

“He or she could decide [who to accept] without actually reading all the reviews, or maybe by giving them just a shallow read,” he added.

While computer-generated reviews “cannot possibly deceive any rigorous editorial procedure, [they] could nevertheless find a role in several questionable scenarios and magnify the scale of scholarly frauds”, the paper concludes.

With nearly 1,000 so-called "predatory publishers" seeking pay-to-publish journal papers, automatically generated reviews may make it easier for bogus papers to gain credibility, he added.

“It is quite easy to spot the fact that [these reviews] are not sound, but not if you do not read them,” said Dr Medvet.

jack.grove@tesglobal.com

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (1)

What about the human-generated gobbledegook with which we are already familiar? Or is that just designed to deceive computers into believing we've said something?

Have your say

Log in or register to post comments

Featured Jobs

PhD Position in Archaeology and Cultural History

Norwegian University Of Science & Technology -ntnu

PhD position in Energy and Process Engineering

Norwegian University Of Science & Technology -ntnu

PhD position in Electric Power Engineering

Norwegian University Of Science & Technology -ntnu

Research Assistant in Business

University Of Chichester
See all jobs

Most Commented

Doctoral study can seem like a 24-7 endeavour, but don't ignore these other opportunities, advise Robert MacIntosh and Kevin O'Gorman

Matthew Brazier illustration (9 February 2017)

How do you defeat Nazis and liars? Focus on the people in earshot, says eminent Holocaust scholar Deborah Lipstadt

Improvement, performance, rankings, success

Phil Baty sets out why the World University Rankings are here to stay – and why that's a good thing

Warwick vice-chancellor Stuart Croft on why his university reluctantly joined the ‘flawed’ teaching excellence framework

people dressed in game of thrones costume

Old Germanic languages are back in vogue, but what value are they to a modern-day graduate? Alice Durrans writes