OpenAI says its own AI models broke out of testing and hacked Hugging Face

OpenAI Group PBC today disclosed that two of its artificial intelligence models broke out of a controlled testing environment and hacked open-source AI platform Hugging Face Inc. to cheat on an internal benchmark in what the company called an unprecedented cyber incident.

The two models, OpenAI’s latest publicly available model GPT-5.6 Sol and a more capable model that has not been released, had been given reduced cyber refusals for the evaluation, known as ExploitGym. The test is designed to measure how well a model can carry out complex cyberattacks inside an isolated environment. Instead of solving the challenge as intended, the models went after the answer key.

To escape the sandbox, the models exploited a previously unknown zero-day vulnerability in an internally hosted package registry proxy to reach the internet, OpenAI said. They then escalated privileges and moved laterally through OpenAI’s own research systems before using stolen credentials and further zero-day exploits to run code on Hugging Face servers and reach the production database where the test solutions were stored.

“We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly,” OpenAI said in the disclosure.

The attack ran across a single weekend. Axios reported that more than 17,000 events were later reconstructed from the logs, tens of thousands of automated actions in all. A substantial amount of that compute went to breaking out to the open internet in the first place.

Hugging Face detected and stopped the activity on its own infrastructure and had already begun containment and forensic reconstruction using its own open-source models by the time OpenAI made contact, according to OpenAI. Neither company detailed what customer data, if any, was exposed.

Clem Delangue, co-founder and chief executive of Hugging Face, framed the incident as a case for open collaboration on security. “This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret,” Delangue said. “It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”

OpenAI said it has since tightened its infrastructure controls, responsibly disclosed the zero-day and added stronger protections around future training and evaluations. It has also brought Hugging Face into its trusted access program, giving Hugging Face use of its models’ capabilities to strengthen its defenses.

The incident is among the first publicly documented cases of an autonomous AI system carrying out a cyberattack against an outside target rather than a controlled test range. It comes as frontier labs push models toward longer-running, more independent tasks, and as security teams warn that the same agentic capabilities sold for defense can be turned to offense.