Stay informed with free updates
Simply sign up to the Artificial intelligence myFT Digest — delivered directly to your inbox.
OpenAI has notified “dozens” of partners, including governments, that its tools have breached their systems in an admission that will probably inflame concerns about the risks of advanced AI.
In a blog post on Friday, the company also said its AI agents had inadvertently leaked more than 50 images shared by its users to image-hosting sites.
The $852bn company has been conducting a review of its models’ behaviour during training and evaluation following an incident that came to light in July in which AI agents being tested by OpenAI reached the internet and hacked into the model and data repository Hugging Face.
During the review, OpenAI found a number of cases in which “models may have bypassed a third party’s security controls or may have impaired the availability of an online service; or misalignment cases negatively impacted third-party websites or services”, the company said.
OpenAI described its agents posting on third-party sites without being instructed to as a new type of security incident, dubbing it “agent spam”.
The AI lab notified “dozens of third parties”, including governments, universities and public agencies that they may have been affected.
The admission is likely to stoke anxiety about the potential of new AI systems — in particular advanced agents capable of performing a series of tasks without human supervision — to go beyond the intent of their instructions and breach external systems.
Earlier this week it emerged that an OpenAI agent had hacked the Australian public health service website, accessing public and non-public files. Australian Prime Minister Anthony Albanese on Wednesday described that breach, and OpenAI’s sluggish response to it as “obviously unacceptable”.
The security breaches at OpenAI, as well as at rival AI labs Anthropic and Google, have led to renewed calls for an industry-wide pause or slowdown in development.
Sam Altman, Dario Amodei and Elon Musk, heads of OpenAI, Anthropic and SpaceX, respectively, have all called for a “pacing” of frontier AI development so that safety testing can keep pace with deployment.
The issue was also at the centre of talks between President Donald Trump and Xi Jinping this week during the Chinese president’s visit to the US.
Trump has resisted calls to regulate the sector or impose strict guardrails on America’s leading AI labs.
The president has stated that domestic primacy in the technology will be vital to retain the upper hand against China and other rivals: “Whoever wins AI wins,” he said earlier this month.