Xi and the Dilemma of AI

The central issue isn’t which nation will dominate AI. It’s whether we will both maintain human control over it.

China and the United States are now competing for AI dominance. But what’s needed is a new mind-set about the meaning of “security” in the emerging world of AI — where the central question should not be who gains AI dominance but whether human beings maintain dominance over AI.

The dilemma is that humans won’t be able to maintain dominance over AI unless China and the U.S. — the two leading AI powers in the world — join together to seek it, but our competition for dominance makes this nearly impossible.

Xi Jinping will be visiting the United States this week. It’s an opportunity for the new mind-set, but will China and the U.S. take it?

Xi recently gave a keynote address at the World A.I., Conference in Shanghai which contained some clues to his thinking. In it he said:

“We should … ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger. We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use, and ensure that AI is always under human control. In the meantime, we should jointly oppose overstretching the national security concept in the field of AI and placing one country’s security over that of others.”

This sounds as if Xi is most concerned about humanity maintaining control over AI.

Yet other officials in China appear more concerned about American dominance over AI.

In an article published September 12 in the state-run magazine China Cyberspace, China’s top spy chief, Chen Yixin, warned that artificial intelligence could pose a direct threat to the Chinese Communist Party’s hold on power. He called for more party control over AI and stricter government oversight — to defend against increasingly sophisticated cyberattacks and misinformation campaigns from “hostile forces,” and to compete militarily with countries like the United States.

Leaders of AI in the United States appear just as ambivalent about whether the most important goal is keeping America’s AI lead over China or keeping AI under the control of human beings.

While Trump and his tunnel-visioned treasury secretary, Scott Bessent, are focused on American dominance, other AI leaders seem equally concerned about human dominance.

Anthropic CEO Dario Amodei is sounding the alarm about AI’s growing ability to build the next generation of AI on its own, without human oversight. This dynamic, called recursive self-improvement, is starting to happen across the industry, including at Anthropic.

“Left unchecked,” warns Amodei, “it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”

Amodei continues: “Given the accelerating rate of AI capability development … in 6–12 months … a swarm [of AI agents] could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.”

But in the same open letter, Amodie insists that democratic nations can only afford to slow down their own AI development if they maintain and widen their technological lead over China.

“Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk. I agree with Secretary Bessent that a Chinese lead in AI would pose grave danger for the United States and the world. The CCP-associated projects will run the alignment risks that US companies are carefully preventing, and even if they avoid those risks, they will be in a position to militarily dominate democracies (for example with AI-driven drones). Thus, a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible, to give us the breathing room we need in order to pace effectively.”

Amodei backs restrictions on selling powerful A.I. chips to China, among other measures to slow China’s progress enough to widen America’s lead.

It’s always hazardous to take the words of political and corporate leaders at face value, so I urge skepticism.

Yet there’s no doubt that some leaders in both China and the United States are aware of how quickly AI is moving toward building, on its own, the next generation of AI — and how dangerous this could be for the human species, regardless of whether humans live in China, the United States, or elsewhere on the planet.

We won’t be able to stop this from happening unless China and the U.S. work together, but our two nations are too suspicious of one another to allow it.

Keep your expectations low for this week’s visit of XI to the United States. But be aware of how high are the stakes for humanity.