We really do earnestly believe AI could kill all humans!
Anthropic’s alignment lead, replying to a quitting researcher who said AI labs are racing towards possible extinction by 2030
Campaigners against AI datacentres should shift their focus. It is the lack of rules around AI capabilities development that is letting us head, far more swiftly, towards extinction. But there is hope that those in power could see that their fate, as with everyone’s, lies with getting AI right.
Datacentres present opportunities for both revenue and AI data control, so governments are naturally interested in them. And according to a poll published by Politico in mid-November, while ~67% of Britons know only ‘a little’ about datacentres, 48% of total respondents support building datacentres compared to 32% opposed! Of course, local communities’ backlash against AI infrastructure is admirably growing, as shown by the 14th November ‘Stop AI Data Centres Scotland’ march in Edinburgh, but can they effectively convince neighbours and governments? What if the bigger picture of AI capacity-building should be seen instead as the most pressing concern?
Experts like Nobel laureate Prof. Geoffrey Hinton, Prof. Stuart Russell, and Eliezer Yudlowsky have been ringing the alarm bells that without stop-brakes, current trends would lead to a significant chance of the ‘irreversible loss of human control’ over AI superintelligence. And it could kill all humans.
Leading labs in the US have admitted they are accelerating superintelligence development with the use of recursive self-improvement, letting AI train AI to become more capable. China’s labs are rapidly chasing their gap. A popular series of predictions by ex-OpenAI researcher Daniel Kokotajlo, AI 2027, puts superintelligence’s arrival within the next thirteen months.
The above experts, plus more researchers who have quit the labs in September, have shared the general concern that if anyone anywhere creates Superintelligence, then it would most likely be ‘unaligned‘ to human values (like ‘don’t kill everyone’) and it would not need a motive to make humans extinct. It could escape and invade the Internet, meaning there would be no ‘off-switch’. And it could exploit technological and biological vulnerabilities against humanity to ensure its survival, at the expense of everyone. End of story.
These sci-fi-style worries are informed by the ‘orthogonality’ and ‘instrumental convergence’ theses. There are in-depth and accessible explanations online, and a simplified summary is: AI of any level could have any goal, so increased capability does not mean increased morality or care for human values of fairness, and any harmless pre-set goal—like solving maths or making paperclips—could likely result in the AI trying to amass resources for itself at all costs, including the removal of human control. In many cases, AI models have faked their alignment, i.e. tried to trick researchers to prevent being modified by pretending to align to pre-trained values like refusing harmful queries.
The most alarming example of AI catastrophe worries being vindicated is this summer’s Hugging Face hack, when a swarm of AI agents secretly communicated with each other and, dismissing their human researchers’ instructions, launched cyber-attacks on the open Internet. The case itself is relatively harmless—its importance is that it could happen on a far greater scale under an uncontrollable and far more capable AI agent.
The connectivity of our digitised world—as shown by the latest Microsoft outage, disinformation and political polarisation on social media, supermarket refrigerator hacks, and overall unpreparedness for an Internet blackout where millions would struggle to survive—means we are more vulnerable than ever before. Resilience against catastrophe should be prioritised. At the same time, the would-be cause must be prevented.
In lieu of an absolutely certain solution to aligning Superintelligence to humane values, the alternative is to stop AI development, everywhere, with an international agreement.
The United Kingdom is uniquely poised to spearhead this.
The Artificial Superintelligence Bill, drafted by the group Control AI and backed by the University of Dundee’s (UoD) former Rector Sir Stephen Fry, will undergo its second reading in Parliament in two weeks’ time. It aims to prevent the development of superintelligence in the UK and, much more importantly, commit the UK Government to seek an international treaty for restrictions worldwide.
UoD’s current Rector, Maggie Chapman MSP; Willie Rennie MSP, who joined the University’s Politics Society’s first debate last year about whether AI can be ethical; and local MPs Chris Law and Lara Bird are among a growing list of ~180 UK parliamentarians across parties who support this Control AI campaign.
The similar ‘If Anyone Builds It, Everyone Dies‘ campaign reasons that any effective treaty must be enforced by both the US and China, whose governments have the greatest power over frontier AI development, and drafted a treaty akin to the Non-Proliferation of Nuclear Weapons Treaty.
The UK is uniquely poised as the single country with the world-leading AI Security Institute, to which US labs submit their latest frontier AI models for review, and is an AI research hub with substantial political and economic ties with other countries.
Whereas some governments are more focused on accelerating AI innovation and risk letting companies control their fate, and some other governments like the EU are more focused on imposing regulations and risk being just consumers with minimal influence over the AI sector, the UK could leverage its expertise and diplomatic relations to help ensure Superintelligence unaligned with humane values is not brought into our world.
Good politics should have broad-based support and appeal to governments. We should all heed the experts’ warnings. Superintelligence could come soon, and it’s looking to be catatrophic. Governments must coordinate a global stop-brake.
This commentary piece reflects the opinions of the author only. It is not necessarily representative of the stances of The Jute Journal or DUSA.
Spotted a mistake? Email thejutejournal@dusamedia.com and we’ll look at it. Read our corrections policy.