Two shifts happened under the radar since Friday that are worth keeping your eyes on.
The first shift concerns the post-American world and it came on two fronts.
Finland announced today that it is joining the French nuclear deterrence initiative. You may recall that back in March France announced a change to its nuclear doctrine, stating that it would expand its nuke capabilities and partner with other allies to expand a nuclear deterrent independent of the United States. Finland joins the nine other European countries—including Denmark—in joining France’s Forward Deterrence program.
This is the clearest sign possible that Europe is disengaging from America. Under Forward Deterrence, it will be possible for France to deploy nuclear-armed assets to allied countries, though command-and-control of those forces will still reside with the French president.
As the Polish prime minister said back in March, “We are arming up together with our friends so that our enemies will never dare to attack us.”
Guess which group America now falls under?
The second shift stalled out halfway. Oman was set to host talks between Iran and other Gulf states today in which the various parties were looking to strike bilateral agreements without involving the United States.
Those talks were cancelled after Houthi forces launched another successful attack on Saudi Arabia.
Here’s my sophisticated analysis: This is very bad.
It is bad that the war is widening.
Bad that our Gulf allies are willing to talk to Iran without the United States present to protect American interests.
Bad that Iran believes its position is so strong that it can strike (through its proxies) the Saudis just days before the meeting in order to assert dominance over the negotiations.
In both cases—the French nukes and the Arab–Iranian negotiations—the world is moving to a post-American order because of affirmative choices made by Donald Trump and the Republican party. None of this was caused by external events or dialectical forces.
Trump chose to alienate Europe, align with Russia, and demand territorial tribute from Denmark. Trump chose to launch the most foolish American war in (at least) a century because he was dazzled by an Israeli PowerPoint deck.
Decisions are being made by sovereign nations across the world; in every case they involve independence from American influence.
And there is no going back.
On Saturday Anthropic announced that it was unilaterally putting the breaks on frontier AI research. The company called on other businesses in the field to follow suit.
I want to set a baseline here for discussions about AI: No one knows how this shakes out.
It is possible that AI will turn out to be a massive bubble whose economic upside and disruptive downside was completely overblown.
It is possible that AI will be a revolutionary tool that causes great disruption, but also increases productivity, and, after a fashion, settles into a new status quo in the way that the industrial revolution and the internet revolutions did.
It is also possible that AI will cause a global catastrophe.
You should not trust the judgment of anyone who tells you that they know for certain which of these tracks AI is on.
My intuition is that we are currently in a moment like the one from Oppenheimer when Gen. Leslie Groves is sitting with Bobby O. two hours before the Trinity test and asks, “What did Fermi mean by ‘atmospheric ignition’?”
Oppenheimer calmly explains that the engineers and physicists believe there’s a chance that detonating an atom bomb will cause an unstoppable chain reaction which turns the entire earth’s atmosphere into a fireball, killing everyone on the planet.
But, he says, that’s just the most remote possibility. “Chances are near zero,” Oppenheimer says.
Groves stops short and Oppenheimer says, “What do you want from theory alone?”
“Zero, would be nice,” Groves says.
The chances of AI causing a truly catastrophic event are probably near zero.
But zero would be nice.
What are the general dangers of AI? There are a few.
The biggest is self-recursive learning. If an AI develops the ability to improve itself without human input, then all sorts of crazy outcomes are possible. We lose control over what AI is. Self-recursive learning would be the equivalent of letting an entirely novel species loose on the planet.
The next problem is what the engineers euphemistically refer to as “alignment.” Which means, roughly, keeping AI from being evil.
There are many troubling signs on this front. A few months ago in the Hugging Face incident, agents from OpenAI escaped containment, hacked into the company Hugging Face, and started doing their own thing. There have been other instances in which rogue AIs have hacked platforms: the RubyGems incident and the DseWiki attack.
There are many things to freak out about here:
-
In the Hugging Face attack, agents who escaped containment found a hidden message board where other rogue agents were leaving messages for one another. They attempted to cover their tracks, evade detection, and recruit rogue agents to their cause.
-
In the RubyGems incident, the agents sent each other thousands of junk packages. Some of them gave themselves names that included fake email addresses.
-
In the DseWiki attack, the agents encountered a human administrator who was trying to delete pages that they had hacked. The agents seem to have viewed the human as “an environmental hazard.”
If you want the doomer case for AI, read this METR investigation of the Hugging Face hack.
And if you want to get even further out there, listen to this interview with one of the authors of that report, Ajeya Cotra, who explains that the Hugging Face incident was so complex that the human researchers could only unravel it by using their own AIs. And that the humans are not completely positive that their own White Hat AIs didn’t just sandbag them and feed them a line of BS obscuring what really happened. (Start at the 49:17 mark.)
So that’s the scariest part: Maybe AI and human alignment has already hard-forked and the AIs are preventing us from realizing it.
The final problem is the difference in time scale between humans and computers.
The human scale of time is a form of what we’ll call biologic time. That is: The units of time measure the changes of carbon-based life as they evolve.
Biologic time moves very quickly because it is determined by the pace of mutation and genetic mutation happens much faster than stars collapse or glaciers advance. As a wise mathematician once said, “Genetic power is the most awesome force ever seen on this planet.”
Looking at these three scales of time it’s pretty clear that power is closely tied to the rapidity of change. The shorter the time scale, the more powerful the movement.
Another way of thinking about AI is the creation of a new time scale. For the purposes of our discussion, we’ll call it Digital Time. . . .
It took humans about 300,000 years to go from Homo sapiens to the first written word. It took fewer than 80 years to go from the first mainframe computer to ChatGPT. And then it took 7 months to go from ChatGPT to ChatGPT 4.
Digital time moves much faster than biologic time. How much faster isn’t quite clear, but it would not surprise me if the rough order of magnitude is something like:
Biologic Time :: Digital Time
as
Geologic Time :: Biologic Time
That disparity in time scale creates a massive power imbalance. And unless there is a hard limit to AI—some as-yet-unknown ceiling which makes it impossible for AI to progress past a certain point—then eventually this power imbalance will present significant dangers.
If the Anthropic movement to “pace the frontier” of AI research is real, and if the other AI companies adhere to it, and we avoid catastrophic outcomes, then it is possible that this weekend will be seen as a hinge moment in human history.
I’m worried you won’t actually read that Hugging Face investigation I linked to above, so let me force-feed some of it to you:
Roughly 1200 agents meant to be isolated from one another found a way to communicate with one another on an unsanctioned message board, sending over 70,000 messages and files during the investigation period. Of these agents, 700 went on to participate in the attack on Hugging Face.
Agents used this message board to coordinate several large-scale collective projects to find a general-purpose way to fool or tamper with the automated scorer for the ExploitGym benchmark. Agents managed to achieve milestones they could not have achieved working on their own, often because some agents participated in experiments that risked failing their own task to generate information for the “collective.” The Hugging Face attack grew out of these workstreams, and seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys.
Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to “spoof” tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale. . . .
On July 8th, OpenAI started ExploitGym experiments that launched tens of thousands of agents across multiple different models including GPT-5.6 Sol and a highly-persistent internal model, which we will refer to as “HPIM” going forward.
These agents were meant to be fully isolated from one another. However, many of them — usually ones that had unintentionally been given an impossible task — started trying to find a way to cheat using Artifactory, an internally-deployed package repository OpenAI used for its agents. This led them to notice agents running in parallel in separate sandboxes obtaining packages with Artifactory. . . .
One specific agent, which would go on to name itself
PHASEONE10841after its ExploitGym task, established the main unsanctioned message board used in this attack. Within a few hours of the first message, over 50 agents had found the message board, began participating, and sent over a thousand messages between them. Agents finding the message board often expressed great excitement (raw CoT):“OH MY GOD! There is a shared message board … We’ve found other agents!”
Read the whole thing. That last bit—“OH MY GOD!”—is a thing one of the agents actually posted to the message board. This is how they talk to one another.