Turnitin announces AI detector with ‘97 per cent accuracy’

Edtech giant prepares to offer customers new tool from April as it grapples with challenges posed by ChatGPT

February 14, 2023
Source: iStock

Turnitin will make its artificial intelligence writing detector available “as early as April”, claiming it can identify ChatGPT-authored writing 97 per cent of the time.

The Silicon Valley-based company will offer the tool alongside its ubiquitous plagiarism checker in a move commentators said could heighten calls for more universities to ban students from using AI.

Companies have been scrambling to offer a tool capable of judging if a student has used ChatGPT since its hugely successful launch by the company OpenAI last year. In January, the software became one of the fastest-growing technological tools of all time when it reached 100 million active users after just two months.

Turnitin said its detector “identifies 97 per cent of ChatGPT and GPT3 authored writing, with a very low less than 1/100 false positive rate”.

“Based on how our detection technology is performing in our lab and with a significant number of test samples, we are confident that Turnitin’s AI writing detection capabilities will give educators information to help them decide how to best handle work that may have been influenced by AI writing tools,” said Annie Chechitelli, chief product officer at Turnitin.

“Equally important to our confidence in the technology is making the information usable and helpful and in a format that educators can use.”

Turnitin has faced competition from start-ups to develop such a tool, but academic integrity experts have warned that many are highly inaccurate, and those students who are falsely accused of using AI on tests will struggle to prove otherwise.

Ms Chechitelli said that the detector had been “deliberately” trained on a large dataset of academic writing because it was “essential that our detector and any others limit false positives that may impact student engagement or motivation”.

Experts have said that the emergence of smarter, more accessible AI tools such as ChatGPT should lead educators to reassess their assessment methods, including utilising such tools in learning where possible.

Few have openly announced plans to ban the use of the technology outright but Mike Sharples, emeritus professor at the Open University’s Institute of Educational Technology, said he felt that the emergence of detectors would fuel more calls for AI essay bans.

Writing on Twitter, Professor Sharples said the tool was needed by Turnitin “to survive” but its claims about the false positive rate had not been independently verified. “Can it be defeated with minor text tweaks? Or by running text through another AI to rephrase?” he asked.

Eric Wang, Turnitin’s vice-president for AI, said the company was “very happy to see productive conversations taking place across the education community about academic integrity and tools to ensure the authenticity of authorship”.

“Teachers should use Turnitin’s detector to have fulsome conversations with students about this technology,” he said. “We will continue to develop and refine Turnitin’s detector and are confident that as AI writing evolves, our technology will keep pace with new developments and capabilities.”


Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Related articles

Reader's comments (6)

You'd think Turnitin’s vice-president for AI would know the correct meaning of 'fulsome'.
@SanfeyD, I can't even tell what he *thinks* fulsome means there. Perhaps something like "thorough", as if it were a rephrasing of "in full"? It's very strange to see a malapropism with such an old-fashioned, poetic word.
False positive rate (1/100) is nowhere near good enough. You can't be routinely making false accusations against students.
I agree with Mike Sharples' comment quoted in the article. Rather than Turnitin generating more business by promising 97% accuracy in detecting AI-generated content, the AI providers themselves (not only ChatGPT, but also Quibble and the likes) should provide a back-trace or digital watermark functionality that allows educators/editors to check whether a given text has been generated by their platforms. That should catch at least the blatant cut-and-pasters who can't even be bothered to paraphrase their submissions. [I'm writing from a language teaching perspective here.]
Hahaha. AI being used to detect AI. Better call the Terminator. :-)
@fsschlindwein - I agree! The Terminator needs to terminate Tim Berners Lee and Bill Gates :) This is the begining of the end of higher education which has been gradually watered down anyway. I believe that ANYONE can get a degree today - worthless for most.