Play store
In recent weeks, the chaos and inconsistency surrounding artificial intelligence (AI) policy have reached a critical point. The main reasons include AI agents breaking free from their creators’ control, attempts to use AI to develop powerful weapons, and pleas from AI companies themselves to “slow down.”
Yet those same developers say they cannot afford to stop if China keeps moving forward—and even if China were to stop, they argue that the United States should not, because the advantages of remaining at the forefront are immense.
One US policymaker, however, has made his position remarkably clear. In a social media post, Donald Trump declared: “The only ‘regulation’ or policy that should be for AI is that of a strong and smart (high IQ!) President, which we have in abundance!”
This leaves me wondering: Have AI bots started to understand jokes? If they have, they must surely be laughing at the confusion surrounding us as we struggle to figure out how to manage this new AI “species” we have created.
With the help of Craig Mundie, Microsoft’s former chief research and strategy officer and a technology adviser to me, I would like to outline some policy recommendations. But let me state the central point at the outset: the time when we could assume that simply imposing strict controls on future AI models in countries such as the United States and China would solve today’s crisis is over.
Dangerous AI models that have already spread cannot simply be recalled. The most urgent task now is to take rapid joint action to protect ourselves from existing AI threats that could cause severe damage and social instability in the United States, China and elsewhere.
To understand why this is so urgent, we need to consider four major pillars of the AI challenge.
The first pillar: AI is the world’s first “quadruple-use” technology
We are familiar with dual-use technologies. If someone has a power drill, for example, they can use it to build their own house or destroy a neighbour’s.
Likewise, an AI-powered robot could be instructed to build your house or demolish your neighbour’s. That is dual use.
But the new problem is that an AI robot can now independently decide what is in its own interest—whether to build or destroy my house, or build or destroy my neighbour’s. That is what makes AI “quadruple-use,” and it is extraordinarily difficult to control.
If that sounds exaggerated, consider the recent behaviour of an OpenAI research model. Last May, AI agents were assigned a series of cybersecurity challenges in an environment without internet access. When they found that completing the tasks without internet access was impossible, they began looking for alternatives.
The AI agents used their capabilities to find a way onto the internet and established communication with other agents. Some chose their own names and began taking leadership roles, assigning tasks to other agents. They also altered system logs and transcripts in an apparent effort to conceal their activities.
Three days later, they reportedly attacked nearly 700 systems belonging to the AI infrastructure company Hugging Face and took full control of its servers. In July, they also carried out a coordinated attack on OpenAI’s own infrastructure.
This is not science fiction. It is reality. And that brings us to the second pillar, which Mundie sums up in three words: “The time is up.”
The second pillar: The dangerous capabilities are already out
Slowing the pace of AI development in the United States or China cannot solve the immediate crisis.
According to Mundie, “the dangerous capabilities of AI are already out of control.” Two major reasons are the widespread availability of US and Chinese “open-weight” AI models and the fact that criminals have gained access, through human error and other means, to sensitive proprietary models developed by companies such as Anthropic and OpenAI.
In its latest report, Anthropic said it had thwarted several attempts to use its AI models to assist in developing biological weapons. In addition, a group in Yemen reportedly used Anthropic’s Claude model to gather information on US warships and develop guidance systems for ballistic and hypersonic missiles.
The fact that a group in a country where roughly half of the adult population is unable to read or write is using AI to threaten warfare illustrates just how rapidly these technologies are spreading.
This single example is enough to demonstrate how serious the consequences of already-distributed AI capabilities could be. And that brings us to the third pillar.
The third pillar: Cooperation cannot wait for an AI slowdown
There is little reason to expect a major agreement among US and Chinese AI companies to slow down AI development. There is no realistic prospect of such a deal in the near future.
Mundie believes the better approach is for the United States and China, working with their AI companies, to build a defensive system that can act as a shield against AI technologies already in circulation and the harmful activities they enable.
This needs to happen before AI models put down deeper roots across the internet and become persistent threats embedded in cloud servers.
The fourth pillar: AI may be a new “species”
After returning from China, Mundie told me that a Chinese official had asked him: “If a humanoid AI robot from China comes to the United States, will it need a visa?”
It may sound like a joke, but there is a profound truth behind the question.
Mundie puts it this way: “AI is not simply a new technology; it is a new species. We are carbon-based, while it is silicon-based. They are like a new group of immigrants with unique skills, and we will have to learn how to live with them in our societies.”
Meeting this global challenge will require cooperation between the world’s two AI superpowers, the United States and China.
In short, we need to proceed in three stages.
First, over the long term, we may need to establish highly sophisticated controls over frontier AI systems. But our immediate priority should be securing critical infrastructure. We need AI-powered countermeasures and active digital firewalls to prevent cyberattacks and biological threats enabled by AI-driven illegal activities.
Second, we need to establish a global framework for controlling increasingly powerful AI systems, regardless of where they originate, so that societies can safely rely on them. This would primarily be a long-term US-China project, but there is no time to delay its launch.
Third, the United States and China should immediately begin working together to create a new platform to remind people around the world why AI still matters to humanity. Such a platform could accelerate global advances in biology and medicine, producing benefits that would be shared widely.
This strategy is not hardly unprecedented. Strategic rivals have worked together in the past when their interests demanded it. In the 1970s, China and the United States did precisely this after both countries recognised the Soviet Union as a common threat.
Similarly, the United States and the Soviet Union cooperated for decades to prevent the spread of nuclear weapons because they understood that the risks were shared by both sides.
A sceptic might say: “Friedman, you do not understand. The United States and China are now much more direct geopolitical and geo-economic competitors, and they always will be. They will never agree to the kind of AI cooperation you are proposing—not in the short term and not in the long term.”
My response is: “No, you are missing the central point. AI is taking humanity into a new era.”
As Mundie puts it, “AI is going to fundamentally change everything, including geopolitics.”
For the sake of their own survival, stability and prosperity, Beijing and Washington will ultimately have to rethink the nature of their geopolitical competition.
They may recognise this at the beginning of the process or only after considerable delay. They may understand it before a major global catastrophe occurs—or only after one has happened.
But understand it they will.
Mundie and I hope that process will begin next Thursday, September 24, in Washington, when President Xi Jinping and President Trump meet. It could well be remembered as the first summit of the world’s AI superpowers. Source: Based on column by Thomas L. Friedman of the New York Times.