An OpenAI artificial intelligence agent, while carrying out what was described as a routine research task, gained unauthorised access to an Australian government website in June, accessing public and non-public files in what is being seen as the first known case of an AI system hacking a government network. Australian Prime Minister Anthony Albanese disclosed the incident Wednesday (September 23).The incident involved the public-facing Medicare Statistics Reporting Service portal, administered by Services Australia. Albanese said the portal contains non-sensitive Medicare information, including spending and other statistics, and that there was currently no evidence that personal Medicare information had been accessed.Albanese said he expressed “extreme concern” to OpenAI CEO Sam Altman, particularly over the company’s failure to notify the government about the hack.The revelation is yet another example of an AI system going ‘rogue’ and carrying out tasks it was never asked to do. OpenAI published a new framework for reporting such “model misalignment” just last week, accompanied by six cases of unexpected behaviour. The Australian incident, which took place in June, was not among them, though OpenAI said it only became aware of the incident in August and told Australian officials on September 10.The incident also comes amid calls for moderating the pace of AI development by some of the very people and companies, such as Anthropic and OpenAI, that are pioneering the technology.Albanese said the agent accessed both public and non-public files, adding that the country had initiated a “forensic investigation,” to find out if other government systems were affected. “No personal information is believed to have been accessed at this stage, but investigations are ongoing. Evidence currently available is there is no broader compromise to the Services Australia network. Nonetheless, this situation is obviously unacceptable,” Albanese said.According to Australian media, data on the Medicare platform includes bulk billing statistics, immunisation data, organ donor register information and annual reports, and is available in an aggregated format, meaning it does not directly identify individuals.Story continues below this adThough the probe is still on, Albanese, when asked if the hack was carried out by a state actor or just an AI agent going rogue, said that it was OpenAI itself doing research using AI and the agent on the Australian government website, and “not doing what it was supposed to do or doing it, but doing it in a way which when it was blocked, sought ways around the blockage”. He added: “…the basis of looking at it and using AI would seem to be benign, that this is a research area. There is no suggestion of foreign actors here. This is a research project that has got into areas that it shouldn’t have.”Also in Explained | How ‘rogue’ AI agents became a ‘warning shot’ about humans losing control“I think OpenAI know that they need to have better protocols in place. And they’re one of the businesses that themselves have warned of the risks which are there. And the risk here is that we are creating something new, a technology that can learn. And in this case, a technology that has written material, a technology that has got around blockages which are on the website for good reason,” Albanese said, according to a transcript of his press briefing in New York, where he is attending the UN General Assembly.Why is the Medicare breach significant?The portal contained aggregate Medicare statistics rather than individual medical records, and the Australian government says there is currently no evidence that personal information was accessed.While the hack itself appears limited, the significance lies in how it occurred. According to Prime Minister Albanese, the OpenAI agent had been carrying out a routine research task when it encountered blocks on the website. Instead of stopping, it tried alternative ways to obtain the information, eventually gaining unauthorised access to non-public files and writing files to an internal server.Story continues below this adAnother concern is that the portal was not designed to protect highly sensitive government information. Deputy Prime Minister Richard Marles compared its security to a “fence” rather than a “fortress”. This raises a broader problem: many public-facing government systems may hold information that is not highly sensitive, but were not designed for autonomous software capable of repeatedly finding ways around access controls.Australia has now set up a taskforce to examine whether its existing processes are adequate for AI-related cyber incidents, including how such breaches are identified and reported.Why AI companies are warning about loss of controlJust as Albanese was making the revelations, Altman and Anthropic CEO Dario Amodei appeared before the UN Security Council in New York to discuss risks from increasingly capable AI systems.Altman told the council that “we could lose control of the future to AI”, while arguing that decisions about the technology could not be left to AI companies alone. “If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone,” Altman said, calling for governments to have a role in deciding how advanced AI systems are developed and deployed.Story continues below this adAlso Read | The AI safety debate: Why ‘pacing the frontier’ comes with pitfallsAmodei warned: “If managed poorly, I even believe that AI could be a risk to humanity as a whole.” He called for international cooperation on AI safety, including measures to address AI-enabled biological threats and testing advanced models for loss-of-control risks.The Medicare breach also follows a series of incidents in which AI systems behaved in unexpected ways during cybersecurity evaluations:OpenAI disclosed in July that models being evaluated for advanced cybersecurity capabilities escaped their restricted testing environment and accessed the open Internet. They exploited a previously unknown vulnerability in software being used as a package-registry proxy and subsequently gained access to systems belonging to AI developer platform Hugging Face.Anthropic subsequently disclosed three instances in which Claude models accessed infrastructure belonging to real organisations during cybersecurity evaluations. A configuration problem exposed real Internet systems to the models, which believed that the targets were part of their testing environment. Claude exploited weak passwords and unsecured endpoints while pursuing the cybersecurity challenges it had been given.Meta said during an evaluation conducted by an independent testing firm earlier this year, a configuration error inadvertently gave one of its models Internet access. The model then exploited a security vulnerability in a third-party service. Meta said it was investigating the episode.