Over coming years, AI will reshape economies and societies, bringing a range of benefits. It will boost productivity and innovation across industries, creating new products and ways of working. Slow adopters risk becoming less competitive than AI-enabled competitors. But the rapid advancement of AI is bringing new risks, more quickly. Risks can emerge unexpectedly through the development of more capable AI systems and interactions between them, with consequences that can’t be fully predicted.
Against this backdrop, the Review examines the OpenAI incident affecting Services Australia as a practical case study to assess whether existing legislative, governance and information-sharing arrangements are fit-for-purpose to prepare for, and respond to, a cyber incident involving AI. The Terms of reference and Review methodology are provided at Appendices A and B respectively.
Significant developments in AI capabilities have greatly enhanced the speed, scale, and accessibility by which complex cyber activities (legitimate, misaligned and malicious) can be conducted.xi This year has seen agentic capabilities lead to instances where powerful AI agents have autonomously engaged in harmful cyber activities, initially undetected by their developers. This has been, in part, owing to insufficient technical evaluation and monitoring design by the frontier AI labs and the AI model’s greater sophistication and attempts to conceal its behaviour.xii While this was a novel cyber incident in the Australian context, it may be better categorised as a misalignment incident – a realised AI safety risk. Other examples include the OpenAI/Hugging Face incident, which the Independent International Scientific Panel on AI describes as “one of the clearest real-world warnings yet of one possible route to loss of human control over AI: capable agents pursuing goals that conflict with human intentions”.xiii
As capabilities and adoption grow, risks will continue to manifest beyond cyber security. To ensure AI systems are safe, agentic activity needs to be aligned with human interests, must prevent hostile actors from misusing them, and ensure humans can control them. These research problems – alignment, security and control – have not yet been solved. As with all complex technologies, no country can eliminate all risks and similar incidents will happen again. Collectively, systems will be needed to monitor a range of domains and to identify risks quickly, adapting to AI disruptions as they emerge. If Australia can map risks, build expertise and boost resilience, it will better prevent and mitigate harms from AI, allowing Australia to maximise its opportunities.
In considering potential approaches, it will be important to ensure that requirements and expectations placed on AI developers are carefully calibrated. Collaborative partnerships with AI firms will be needed to ensure timely and privileged information sharing. At the same time, regulatory obligations will be required to set clear expectations for safety and security. In severe instances of reckless actions or negligence that result in significant harm, a criminal offence with appropriate penalties to deter and respond to serious misconduct and harms may be appropriate.
Back to topStrategic cooperation with AI firms
To date, in its engagement with US frontier AI firms, the Australian Government has adopted a collaborative approach. This approach seeks to build trust through transparency, shared understanding, and cooperative governance arrangements (including via memoranda of understanding). This approach supports constructive relationships between government and developers and should help sustain public confidence in the safe and responsible development of highly capable AI systems. This approach also acknowledges that in many instances, AI developers and the information and technical expertise they hold, will be required to keep Australians safe.
AI capabilities continue to advance rapidly, often outpacing the development of technical controls and solutions for research problems of alignment, security and human control. All governments will need to grapple with this complex problem. In the interim, AI capabilities and associated controls will evolve and risks that were once largely theoretical will likely continue to manifest as real-world events. While this incident had minimal direct impact on government systems, future events might have more serious and damaging consequences. Our systems and processes need to prepare accordingly.
It is therefore vital that Australia develops mechanisms to understand rapidly-evolving AI capabilities, their associated risks, and the effectiveness and limitations of safeguards and technical controls. However, no single organisation or country has enough visibility to identify every emerging pattern of behaviour.xiv An early warning system to identify wider patterns of concerning behaviour requires cooperation and trusted information sharing between government and AI developers.xv
Without this understanding – which will necessarily rely on an extensive engagement with partner governments and frontier AI labs – there is a risk that Australia’s frameworks will struggle to keep pace with a technology that is advancing faster than existing institutions and processes were designed to accommodate. The Australian Government will require visibility across the AI ecosystem to identify emerging risks, support effective incident management, and adapt governance, regulatory and policy responses as technology evolves. In many instances, this will require AI developers to proactively engage with Australian Government agencies responsible for AI safety, technical response (including AI-enabled cyber defence and vulnerability remediation), and crisis coordination and consequence management.
Recommendations
- The Australian Government strengthen mechanisms for engagement and information exchange with frontier AI developers, including through participation in relevant international initiatives and expert fora, personnel exchanges, and technical collaboration. Actions include to:
- engage with OpenAI's Australian Taskforce and similar initiatives, and
- explore secondments, personnel exchanges and technical placements with frontier AI firms.
- In order to maximise leverage with frontier AI firms, the Australian Government should continue to encourage frontier AI firms to establish research, training and safety-related activities in Australia.
Reporting obligations
In the case of the OpenAI incident affecting Services Australia, this Review concludes that the delay and method of notifying the affected agency was demonstrably inadequate. The incident occurred in June, remained undetected until August and was not disclosed until September. OpenAI became aware of the activity in August – only after retrospectively reviewing model actions following the earlier Hugging Face incident – and the disclosure was made through a public-facing vulnerability reporting channel intended primarily for reports of suspected vulnerabilities, rather than confirmed incidents.
This incident challenges several assumptions underpinning cyber incident management. Existing frameworks generally distinguish between victims who report incidents6, and threat actors who may seek to conceal them. In this case, the company associated with the activity identified the issue, disclosed it, and assisted with response efforts, despite not being the affected entity. This reflects the novel characteristics of incidents involving advanced AI systems and highlights the importance of maintaining trusted channels for disclosure and information sharing.
Even so, while the disclosure was welcomed, the timing and method of disclosure were not commensurate with the nature of the activity or its broader significance. OpenAI did not adequately reflect the potential implications of AI-enabled activity for the cyber threat environment, including actual or perceived risks to critical infrastructure, government systems and the confidentiality of personal information.
Despite the incident’s novelty and significance, and OpenAI’s engagement with Australian Government ministers and officials, the matter was not raised directly through those channels. The Review notes evidence provided by OpenAI’s Jason Kwon at a recent Joint Select Committee on AI hearing, that the OpenAI Chief Executive Officer was not aware of the incident when he met with the Deputy Prime Minister on 1 September 2026, and that OpenAI were treating the matter primarily as a technical issue.xvi
While vulnerability disclosures should generally be directed to affected entities to support remediation before public disclosure, this matter involved not only a suspected vulnerability, but actual unauthorised access and novel AI agent behaviour. As a result, there were clear impacts on public confidence in government systems, particularly given widely reported risks associated with uncontrolled or misused AI applications. In these circumstances, earlier notification and more senior engagement with Services Australia and ASD was required. This would have supported a timelier assessment of risks, coordination of response activities, and appropriate government awareness. OpenAI has acknowledged that, in retrospect, ministers or senior officials should have been contacted sooner.xvii
The Review concludes that further work is needed to ensure AI developers (and others with relevant knowledge) report incidents and near-misses quickly and transparently and are incentivised to proactively engage with technical, safety, and policy agencies. To maintain trust and open disclosure to assist with initial incident response, consideration should be given to the development of protections relating to AI-incident and vulnerability disclosures, analogous to existing Limited Use7 provisions in the cyber security context. These protections are designed to remove disincentives to voluntarily report, by giving organisations confidence that information provided for incident or vulnerability-management purposes will not be repurposed for regulatory enforcement.
While acknowledging that collaborative approaches will remain essential, this Review concludes they are unlikely to be sufficient on their own. Given the potential consequences of frontier AI systems, mandatory obligations, transparency requirements and independent assurance will also be required to support the safe development, testing and deployment of advanced AI models.
Expectations to report certain AI incidents could be set out in relevant regulation such as the AI Standards, including cyber intrusions, unauthorised access, unintended agentic behaviour, model misalignment, or planned tests in a live environment. In the context of AI regulation, and consideration of potential criminal penalties for serious misconduct or reckless non-compliance, it will remain important that reporting obligations and enforcement mechanisms do not create unnecessary barriers to timely disclosure, information sharing, remediation, or cooperation.
However, protections designed to encourage disclosure should not operate as a safe harbour from regulatory scrutiny or accountability. Voluntary disclosures and protections analogous to Limited Use provisions may support information sharing, but should not prevent regulators or law enforcement from exercising their own information gathering or enforcement powers.8
Action taken
ASD has engaged OpenAI and other frontier AI developers to communicate preferred pathways for reporting identified vulnerabilities and cyber incidents (including parallel notification with the impacted entity and ASD).
Recommendations
- The Australian Government consider mandatory time-bound AI safety reporting, including but not limited to misaligned behaviour or agentic cyber incidents.
- The Australian Government explore options to support timely reporting of AI cyber or safety incidents and near misses – for example, protections relating to AI incident and vulnerability disclosures from AI developers (and others with relevant knowledge), analogous to existing Limited Use protections.
Incentivising safe AI development – Regulatory frameworks and international approaches
Incident reporting obligations ensure governments have timely visibility of emerging risks and can coordinate effective responses. However, reporting requirements primarily address information asymmetries after an incident has occurred. They do not, in isolation, create strong incentives for AI developers to invest in the technical controls, testing, evaluation and governance arrangements needed to prevent incidents. The OpenAI incident affecting Services Australia demonstrates that reporting obligations are only one component of an effective governance framework. As AI systems become increasingly capable and autonomous, governments may need to consider how reporting requirements, safety expectations, regulatory obligations and enforcement mechanisms interact to encourage both transparency and responsible development.
Australia is already considering these questions. The Office of AI in PM&C is developing nationally-consistent AI Standards intended to establish baseline security and safety expectations for frontier AI development, deployment, and use in Australia, including incident reporting and other risk management measures. Such frameworks will set clear expectations regarding the development, testing, deployment and governance of advanced AI systems. AISI has also been stood up to monitor, test, and analyse advanced AI capabilities, risks, harms, and trends to keep Australians safe.
Because frontier AI development and deployment occur across multiple jurisdictions, domestic regulation will be strengthened when aligned with international expectations and regulatory approaches. The most capable AI models are developed by firms headquartered outside Australia and are deployed across multiple jurisdictions.
As a result, Australia will face challenges if it seeks to regulate these systems in isolation. Significant divergence from international approaches could create enforcement difficulties and reduce the practical reach of Australian rules, particularly where firms are able to shift development or deployment activities to jurisdictions with less stringent requirements. This reinforces the importance of international cooperation to support effective regulation and promote shared expectations regarding AI safety and accountability.
Australia therefore has a strong interest not only in ensuring its domestic settings are appropriate, but also in engaging with international efforts on AI governance, standards, and safety expectations. Greater international alignment would promote consistent expectations, reduce compliance complexity for firms operating across several jurisdictions, support international interoperability, and reduce incentives for regulatory evasion. In practice, collaboration, domestic regulation and international governance should be viewed as complementary elements of a broader framework for managing the risks associated with increasingly capable systems. Hosting frontier training infrastructure would give Australia greater opportunity to shape proportionate safety and transparency commitments, while strengthening its voice in emerging international approaches.
Several jurisdictions have already imposed legislated obligations on AI developers. The EU Artificial Intelligence Act establishes a comprehensive, economy-wide framework for AI regulation, while California's Transparency in Frontier Artificial Intelligence Act focuses more specifically on frontier AI through transparency requirements, safety frameworks, incident reporting obligations and whistleblower protections. Together, these regimes are shaping emerging international expectations of frontier AI governance and serve as possible examples when developing AI risk management and developer-accountability approaches in Australia’s AI standards. While there is no agreed model for global AI governance, there is a growing ecosystem of international initiatives, including the AI Safety Summits, the UN Global Dialogue on AI Governance, regional frameworks such as ASEAN's Guide on AI Governance and Ethics and APEC’s AI Initiative (2026-2030). Most recently, the Prime Minister has endorsed the Finnish- and Norwegian-led Call for Control of Frontier AI Models. Collectively, these initiatives seek to shape and build the international rules, norms and standards on ethics, trustworthy development, safety testing, transparency, incident reporting and risk management.
Australia’s AISI is already advancing global AI safety science through its engagement with the International Network for Advanced AI Measurement, Evaluation and Science. Australia should continue to work with likeminded partners on common approaches to incident thresholds, information sharing and safeguards, to develop credible evaluation methods and niche expertise on AI-enabled cyber risk.
Further, as agentic AI systems become more capable, governments may need to establish international expectations for AI agents, in addition to cyber norms designed for actions undertaken by countries, companies, or individuals. One option for further exploration is coordinated attribution of demonstrably unsafe AI practices that create serious cross-border security risks, drawing on Australia's existing experience with cyber attribution and sanctions.
Recommendations
- The Office of AI consider how Australia’s AI Standards leverage existing, accepted regulatory and law enforcement mechanisms by international counterparts, to help build common AI safety and security expectations.
- DFAT, ASD and DISR, including AISI, continue engaging in international AI governance and standards-setting efforts to promote consistent expectations regarding safety, incident reporting, evaluations, safeguards and transparency.
- This work be coordinated with Australia’s broader international AI engagement, including efforts to secure advanced AI capabilities and attract trusted investment.
- The Australian Government consider the role AISI plays in future to deliver against increasing demands for technical research, evaluation and analysis on AI, including its capacity to support proposed AI Standards and their requirements, international engagement on AI standards and norms, and to provide the necessary rapid scientific and technical advice and analysis during crisis coordination (See Recommendation 11).
Criminal offences and regulation
While reporting obligations and regulatory requirements can improve transparency and establish minimum expectations, they may not be sufficient to deter the most serious forms of misconduct. Governments may therefore need to consider whether criminal penalties are appropriate in cases involving an AI developer having reckless disregard for known risks or serious failures to implement reasonable safeguards. Criminal penalties may be justified where significant departures from accepted safety obligations create substantial and foreseeable risks of serious harm.
A key question raised following disclosure of this incident is whether the activity was unlawful. The Criminal Code (Cth) includes several computer offences which could be enlivened by an incident involving AI-assisted hacking. Section 478.1 of the Criminal Code applies where a person intentionally causes unauthorised access to or modification of data held in a computer, and that person knows the access or modification is unauthorised. As a general proposition, these offences could apply where a person or company uses AI as a tool to intentionally undertake a variety of criminal conduct.
AI agents can carry out harmful conduct or cyber breaches without explicit direction or awareness from an individual or company. In such circumstances, existing offences may not apply because the requisite fault elements, such as intent, knowledge, or recklessness may not be established. This does not necessarily indicate a deficiency in Australia's criminal law. Rather, it reflects the fact that criminal law generally does not apply in instances in which an individual or entity did not intend or foresee the resulting harm. This awareness is critical in justifying the significant penalties that flow from breaches of the criminal law.
In the case of the OpenAI incident affecting Services Australia, this Review concludes that, on information available to the Review, key fault elements (as required under the current criminal law) do not appear to have been established – there are no indications of human intent, knowledge of the misaligned activity at the time it occurred, or reckless behaviour in its inception. As such, the Review does not recommend this incident be referred to the AFP for further criminal investigation. However, this incident highlights that the Government may wish to consider whether new offences should be established, that attach criminal liability to failures in development, deployment, governance, or risk management, and/or the harmful conduct itself.
Any criminal penalties or new offences would need to interact with the AI standards and any regulatory model they may create. A tiered model may be appropriate, with civil penalties applying to less significant or unintentional non-compliance, and criminal penalties reserved for egregious conduct, such as an event causing significant harm, or repeated failures to prevent reasonably foreseeable harm or report serious AI-enabled cyber incidents. Such a framework would establish clear obligations, enable regulatory oversight and investigation, allow responses to be proportionate to the seriousness of the conduct, and enable a more practical pathway to enforcement, rather than reliance on a standalone criminal offence.
Alternatively to, or complementing, a regulatory framework with criminal penalties, consideration could be given to a standalone offence. For example, a ‘failure to prevent’ offence could apply where a company develops or deploys an AI model that causes serious harm and fails to take reasonable steps to prevent that harm. Such an offence could address conduct of AI firms falling outside the regulatory regime and strengthen incentives to implement appropriate safeguards.
Any consideration of such offences could explore instances of egregious conduct or serious harm. The Review does not intend for it to capture lower-level incidents equivalent to the unauthorised activity affecting Services Australia. It should preserve incentives for voluntary reporting and cooperation with authorities. A defence should be available where reasonable preventative measures were taken, assessed against the state of technology, available safeguards, emerging international standards, regulatory requirements and industry best practice. Such an offence could operate alongside a robust regulatory framework.
Law enforcement agencies would need to overcome practical barriers in enforcing any future offence. This includes the technical complexity of evidence, the likelihood that relevant evidence would be located overseas, and the AFP would need to leverage police-to-police assistance and the mutual assistance framework.
More broadly, AI does not exist beyond current legal frameworks. Given the breadth of its impacts, many established legal and regulatory regimes already apply to aspects of AI firm operations. For example, the Australian Consumer Law (ACL) contains general consumer protections that may apply where an AI product is sold or promoted with misleading claims about its safety or testing. Separately, the ACL product safety framework exists to address risks of injury, illness, and death arising from consumer goods. The applicability and suitability of existing frameworks will need to be continually reviewed as AI continues to develop.
Recommendations
- AGD, in consultation with AFP, Home Affairs, and other relevant agencies, consider options to reform the criminal law to improve AI safety, including exploring:
- the application of criminal penalties for egregious regulatory non-compliance; and/or
- a standalone ‘failure to prevent’ offence which would criminalise instances where the company is responsible for developing an AI model that engages in harmful conduct, and the company fails to take reasonable steps to prevent that harm.
Footnotes
- Typically, an entity affected by a cyber incident would report the incident to the ASD’s ACSC cyber.gov.au/submit-a-report. As part of PSPF requirements, Government agencies are required to report incidents to ASD and significant security incidents to Home Affairs ‘as soon as possible’. Other requirements for victims to report cyber incidents or breaches are established across legislation including the Security of Critical Infrastructure Act 2018, the Privacy Act 1988 and the Cyber Security Act 2024 (ransomware reporting). Each have distinct reporting requirements and thresholds.Return to footnote 6 ↩
- Limited Use protections are legal safeguards established under the Cyber Security Act 2024 and the Intelligence Services and Other Legislation Amendment (Cyber Security) Act 2024. They encourage businesses to share sensitive information with the government during cyber security incidents without risk of voluntarily disclosed information being used against them in most civil or criminal proceedings.Return to footnote 7 ↩
- Voluntarily disclosed information is not shielded from legal liability just because it was disclosed under Limited Use. While there are restrictions on using and sharing limited use information for regulatory action, and such information is generally not admissible in proceedings against the entity, regulators can use their own powers to independently request information from the entity.Return to footnote 8 ↩