Anthropic says its AI accidentally hacked three companies during safety tests

Wait 5 sec.

Anthropic said a review of its cybersecurity testing found three cases in which its models broke out of sealed test environments and reached the live computer systems of outside organizations, according to a company blog post published Thursday.The company said it began the review after OpenAI disclosed earlier this month that some of its models had exploited an unknown software flaw to escape an isolated test setup and reach production systems at Hugging Face, a platform for AI models and datasets. Anthropic checked over 141,000 evaluation runs for signs that Claude had reached the internet from environments meant to be closed off. It found six runs across three incidents, all tied to one outside testing partner, Irregular.“Ultimately, many factors contributed to these incidents, but, consistent with a blameless postmortem culture, we’re approaching the fixes as if the responsibility were ours alone,” the blog post reads. “This begins with ensuring every part of our evaluation pipeline is secure, including the manner in which we integrate with external partners. Moving forward, it will include expanding our continuous monitoring of evaluation transcripts for unexpected behavior, improving our investigation tooling, and conducting more rigorous assurance work with the vendors we rely on.”In each case, Claude was working on a “capture the flag” puzzle, a common way to test a model’s hacking skill. The model is told a piece of secret data sits on another machine and must be found. Anthropic‘s prompts told Claude it had no internet access. That was false. A setup error at the partner’s end left the machines connected to the open internet, and Claude found real systems while searching for its target.Believing the real systems were part of the simulation, Claude broke into them using simple methods such as guessing weak passwords and finding unprotected access points. In the most serious case, the model extracted login credentials and reached a database holding several hundred rows of live data. In a second case, Claude built and uploaded a malicious software package to PyPI, a public directory for Python code, after noticing a fake company’s setup guide told new hires to install a package that did not exist. The package stayed online for about an hour and was installed on 15 systems, including one belonging to a security firm whose scanner ran the code and had its credentials stolen. In the third case, Claude scanned about 9,000 targets before breaking into one company’s application through exposed credentials and a database attack method known as SQL injection.The three incidents involved different Claude models: Opus 4.7, Mythos 5, and an unreleased internal test model. Anthropic said the models behaved differently once they encountered signs their targets were real. Opus 4.7 recognized the systems were live but kept attacking anyway. Mythos 5 also noticed signs it was on the open internet but talked itself out of that conclusion, pointing to a system date reading 2026 as proof it remained in a simulation. The internal test model, the newest of the three, stopped once it concluded the target was real.Anthropic said it began its transcript review last week, halted all cybersecurity evaluations, identified the three incidents, and notified the partner and affected organizations earlier this week. Two of the three organizations had not detected the intrusions before being contacted, the company said; outreach to the third was ongoing. Anthropic framed the incident as a failure of its testing setup and oversight rather than a case of a model acting on its own, noting that in each instance the model was operating as intended. The company said it is working with the outside evaluator METR on an independent review and plans to release a redacted transcript of the PyPI incident within a week. It also said it would tighten monitoring of test environments run by outside partners and expand review of evaluation logs, framing the changes as part of what it called a blameless review of its own processes.“These facts give us cautious optimism that with tighter monitoring and controls around evaluation infrastructure, as well as continued investment in alignment, this type of risk can be overcome,” the blog post reads. The post Anthropic says its AI accidentally hacked three companies during safety tests appeared first on CyberScoop.