Royal Society urged to declare AI ‘emergency’

The Royal Society should declare the pace of artificial intelligence development an “emergency”, mathematicians have urged, amid escalating alarm about the threat the technology potentially poses to humanity.

On 16 September, 42 fellows and foreign members of UK’s science academy, signed an open letter to Royal Society president Paul Nurse expressing their “extreme concern about the pace of development of AI and the implications of this for humanity and civilisation.” Since it was posted, the number of signatories has already risen to almost 200 researchers.

Recently, a model developed by the AI firm OpenAI solved one of the seven millennium mathematical problems, called the Navier-Stokes problem – a question at the frontier of research that humans have worked on for around 90 years.

Models created by firms OpenAI and Anthropic “are now operating at the level of the top human mathematicians in many parts of the subject and we must assume there is a significant chance of them developing superhuman abilities within a similarly short timeframe,” the letter reads.

Despite the “profound implications for mathematics”, the authors say they are writing to warn that “it is highly likely that these systems are comparably strong in other technical domains such as…. cyber security, autonomous weapon control [and] development of biological and chemical agents and the targeted spread of misinformation. Moreover, we must assume that the rate of change of their capabilities will be similarly rapid in these areas.”

Concerns have been rapidly escalating over the existential threat posed by the pace of development of AI after a researcher at Anthropic, Jacob Coxon, quit and warned that there was a 10 per cent chance that AI could wipe out humanity by the end of the decade.

Subsequently, the head of Anthropic, Dario Amodei called for the pace of AI development to be slowed down. His call was echoed by OpenAI chief executive Sam Altman and Elon Musk. The latter, whose Grok AI has run into repeated controversies, remains a Royal Society member after the body resisted widespread calls last year from other fellows to expel him for his role in defunding some areas of US research via the Department of Government Efficiency (DoGE), which he is believed to have unofficially headed.

“We have all seen the concerns [about human extinction] raised by former employees of the large AI companies,” the open letter says, adding that the “rise in capabilities of these models in our domain of expertise persuades us that such statements must not be dismissed as ‘hype’.”

“By the time the situation becomes obvious to the wider public it may be too late to act. We believe this is an emergency and call on the Royal Society to use its influence to convey this view to government and the media.”

frances.jones@timeshighereducation.com