Summary:

• Playing catch-up: To shore up its sovereignty in the AI era, the EU wants to triple its data centre capacity in the next five to seven years.

• House of CADA: The Cloud and AI Development Act would fast-track data centre permitting – but wouldn’t require community consultation when setting acceleration zones.

• So what: Without local buy-in, Europe risks repeating the US experience, where about $130 billion in data centre projects were blocked or delayed in a single quarter this year.

• Recommendations: The European Parliament and Council should amend CADA to require early participation (a say, not a veto), a share of the benefits for host towns, and enforceable commitments.

Subscribe now

In February 2026, the German town of Gross-Gerau rejected a proposal by Vantage Data Centers to build a €2.5 billion data centre. Environmental and tax concerns played a part in the decision, but local campaigners indicated that had Vantage approached the town earlier, the result might have been different.

Europe hosts around 5% of global AI compute. The Commission’s AI Continent Action Plan admits that Europe’s competitiveness and sovereignty depend on building more capacity at home. A recent proposal by AI policy experts and Nobel laureate economists argues Europe should raise its compute share to 15% by 2030.

Lessons from America

In the US, local opposition blocked or delayed at least 75 data centre projects worth about $130 billion in the first three months of 2026, roughly matching all of 2025. As grassroots opposition groups double in number, so does their political weight: over 530 US local laws sought to ban or restrict data centres as of July.

For America, these losses are a fraction of its data centre pipeline, which is now approaching $2.3 trillion. Europe has seen less opposition so far, but it has also built far less; as it scales up, a similar backlash could seriously set back its ambitions.

America’s data centre rollout has largely treated local communities like an obstacle to manage quietly, with non-disclosure agreements and applications filed as ‘light industrial’ or ‘technology park’. Communities have caught on to the playbook: nearly half of Americans now say data centres are mostly bad for quality of life nearby, up from 30% in January.

However, in some places, Europe has already imported the same playbook. In the Dutch town of Zeewolde, councillors were briefed for two years on a project codenamed ‘Tulip’ before learning it belonged to Meta.

Europe needs towns on board. This doesn’t mean local vetoes over data centres, but rather a say on where and how they’re built.

CADA: built for speed, not buy-in

The Cloud and AI Development Act (CADA), currently before Parliament and Council, aims to triple EU data centre capacity within five to seven years. It would require member states to designate ‘data centre acceleration zones’, where authorities must decide on a permit within 12 months of a developer filing a complete application.

Power and permits are the biggest bottlenecks to Europe’s data centre ambitions, and CADA rightly targets them. But local backlash could become the next.

Two EU laws give the public a say over data centres, but only under conditions. Under the environmental impact assessment directive, residents must be consulted on specific data centre projects before they are approved, but only if national authorities decide an assessment is needed; data centres do not necessarily get one. Under the strategic environmental assessment directive, when a government publishes a draft plan for where development can go, residents can comment on its environmental effects.

With CADA the decision would move from the project to the zone: authorities would assess each acceleration zone once and issue a zone-wide permit (Art. 13(2)–(4)). CADA doesn’t guarantee communities a say when the zones are drawn; residents are not included among the bodies that member states must involve (Art. 10(4)). If residents are consulted on a specific project at all, the zone and its permit are already settled.

CADA’s only mention of the public is incidental: each member state’s permitting office may also hand out information ‘with the aim of increasing public acceptance of the data centre project’ (Art. 12(2)(e)).

Renewable energy projects get a different deal

The 2023 revision of the EU’s Renewable Energy Directive, RED III, requires member states to create ‘renewables acceleration areas’ with a 12-month permitting clock, and consult the public when drawing them.

In December 2025, the Commission proposed to amend the renewables directive again. For renewable projects above ten megawatts, member states would have to ensure a share of the benefits reaches local communities. The Commission’s reasoning is that lack of transparency and insufficient community involvement cause delay and litigation, while early public involvement is ‘a clear success factor’.

Cutting veto points without involvement leaves the reasons to say no intact; they resurface in court and at the ballot box.

Under the Commission’s proposals, residents near an 11-megawatt solar farm would receive a share of its benefits, while those near a €2.5 billion data centre would get nothing.

How to improve CADA

As they negotiate CADA, the European Parliament and Council should reuse three rules the EU has already written for renewables:

1. Give communities an early say

CADA should use RED III’s language: mandatory participation at the zone-drawing stage for the public affected or likely to be affected.

A 2025 study of US large-scale solar projects found that local communities that reported more engagement in the planning process viewed their projects more positively. The same study reported that most respondents expected the public to offer input on such projects, not make decisions.

2. Share the benefits with host towns

CADA should give local communities a share of the benefits, as the December 2025 renewables proposal would.

Germany’s cabinet has approved a bill directing 90% of trade tax generated by data centres to the host municipalities, as Germany already does for wind and solar.

A recent study suggests that benefit-sharing works: once German towns shared in wind turbines’ trade tax revenues, the decline in local support for renewables associated with new turbines was at least a third smaller.

3. Enforce what’s agreed

Without enforcement, towns will be skeptical of promises made by data centre developers. As drafted, CADA requires member states to impose penalties on cloud providers that breach cloud sovereignty rules (Arts. 20–24). It should require similar penalties to ensure developers keep their promises to communities.

Consult early, build fast

Benefits alone are unlikely to be enough. Denmark’s wind compensation schemes left some residents feeling bribed rather than recognised. Participation and benefit-sharing need to go together: one gives towns a voice, the other a stake.

Listening to communities need not add a new stage. CADA already requires member states to involve local authorities when drawing zones (Art. 10(4)), and residents could join that same process.

Increasing its compute share can improve Europe’s options in the AI race. But any meaningful data centre build-out will run through communities. America is learning what happens when it builds around them. Europe should build with them.

Summary

• Concerning capabilities: AI systems can now provide expert-level guidance relevant to making biological weapons, and have generated genomes for viruses not seen in nature.

• Inadequate safeguards: Many AI models have no safeguards preventing biological misuse, and the safeguards that do exist can often be jailbroken.

• Secure the supply chain: Bad actors need genetic materials to build a bioweapon. This makes the physical supply chain a critical place to mitigate risk.

• Recommendations: Governments should require companies producing synthetic DNA to screen orders, verify their customers, refuse and report flagged requests, and undergo independent evaluation to ensure compliance.

Subscribe now

Biological weapons have long been a national security concern, and AI is on track to make building one easier. In June 2026, the CEOs of the leading AI companies, life sciences researchers, and national security experts published an open letter urging the US Congress to act.

On 30 September, California became the first jurisdiction in the world to mandate gene synthesis screening. This requires providers of synthetic DNA to screen orders for dangerous sequences and verify their customers before shipping them.

Other jurisdictions must now build on California’s example.

AI is changing biosecurity risk

Building a biological weapon is not easy. Replicating even a known virus requires its genome sequence, specific materials such as synthetic DNA, the expertise to assemble those into a replicating pathogen, and avoiding the attention of security agencies. These barriers are a common explanation for why bioterrorism attacks have been rare, particularly from unsophisticated actors.

AI is starting to erode these barriers.

Leading AI companies now warn that their models, without sufficient safeguards, could meaningfully help a non-expert create a biological weapon. AI systems can provide expert-level guidance relevant to biological weapons development, and outperform PhD-level virologists on questions about technical laboratory procedures.

AI-assisted design of new pathogens is also progressing. Biological foundation models have generated genomes for viruses (bacteriophages) not previously seen in nature, with some replicating faster than the natural virus they were modelled on.

While we should not overstate the current risk, the pace of change is concerning. About 18 months separated an AI agent finding its first zero-day cybersecurity vulnerability and Mythos discovering thousands in major operating systems. Where will biological capabilities be 18 months from now?

Model safeguards are not enough

We need to prevent AI models from providing information that could help someone develop a biological weapon. Techniques like classifiers, in-built guardrails, data filtration, and red-teaming can all help, and the companies producing the most advanced AI models already implement some of these safeguards.

However, model safeguards can be bypassed. In an EpochAI survey, only 3% of over 360 biological AI tools had any safeguards, and where they exist, jailbreaks are common. Anthropic recently noted it is “likely impossible to completely prevent universal jailbreaks”. It is easy to remove safeguards from open-weight models, and those models can’t be withdrawn from circulation.

We need multiple layers of defence.

Targeting the physical supply chain

A bad actor with AI guidance still needs physical genetic materials, like synthetic DNA, to build a bioweapon.

Gene synthesis screening mitigates risk at this chokepoint and is one of the best-understood, least disruptive ways to prevent biotechnology misuse. The core idea is: before shipping an order, synthetic DNA providers check for ‘sequences of concern’ and verify the customer. Sequences of concern are genetic sequences that could be used to create or enhance a dangerous pathogen.

Many major providers already screen voluntarily as part of the International Gene Synthesis Consortium, but without enforcement, providers may choose not to participate.

An effective screening regime needs four components

1. Providers should screen orders and verify customer identity. Screening should apply to sequences that are above a minimum length threshold (e.g. 50 nucleotides). Forward-looking policy should cover ‘benchtop’ synthesis devices – machines that let users synthesise DNA themselves – in addition to commercial synthesis orders. California’s new law does exactly this.

2. Providers should check orders against an evolving database of sequences of concern. The database should combine national and international control lists, such as the US Select Agents list and the Australia Group Common Control Lists. It should also include sequences known to contribute to pathogenicity or toxicity, but which aren’t on those lists. And given that a database of dangerous sequences can be an information hazard, it should have cybersecurity and access controls.

Governments should designate a body responsible for both determining what ‘counts’ as a sequence of concern and updating the database as this evolves. Independent not-for-profits like IBBIS already develop shared databases and work with governments, industry, and scientists to maintain them.

Over time, screening should move beyond flagging sequences similar to known threats and toward flagging sequences that encode a dangerous function. This is particularly important as AI becomes more adept at designing dangerous sequences that don’t look like known threats. But while function-based screening matures as a technology, similarity-based screening is a good start.

3. Providers should investigate, refuse, and report flagged orders. A flagged order should trigger heightened customer verification, where providers confirm the customer has a legitimate use for the relevant sequence. If the provider still has concerns, they should refuse the order and report the customer to relevant authorities. Reporting matters because the prospect of being reported can have a deterrence effect. Providers should also be required to keep records of orders and customers, so a concerning transaction can be traced and investigated later.

4. Governments should verify that providers are complying. The minimum approach would be to require providers to self-attest that they screen in accordance with certain standards. Stronger approaches involve independently evaluating providers and their screening methods, with penalties for non-compliance.

Getting to widespread screening

Beyond California, international momentum is building.

In the US, the Biosecurity Modernization and Innovation Act proposed in January 2026 would mandate screening for all providers. In the last 24 months, the EU and New Zealand have similarly proposed Acts that would introduce screening obligations, and the UK has published voluntary screening guidance.

While these steps are positive, legislation is not yet in force and compliance remains voluntary.

How a country implements screening depends on its policy environment and synthesis industry. A country needn’t be a major industry player like the EU.

In fact, countries with less of a domestic synthesis industry may find it easier to act – given they can regulate imports rather than legislate and negotiate with an established domestic sector. Australia, for example, does not have a significant domestic industry, but is well placed to mandate screening through its existing import scheme – without new legislation.

One jurisdiction acting will help reduce risk, but is not enough. We need widespread screening to stop bad actors slipping through the gaps.

AI is not enabling catastrophic biological harm yet. But we must be prepared for where we might be in six months, one year, or five years from now.

Summary

• Free to download: Chinese AI models Kimi K3 and GLM 5.3 are just months behind American frontier models in cyber capabilities, and are freely available on the internet.

• Missing rules: Neither China nor the US has set standards that developers must meet before publishing the weights of frontier AI models.

• Two to tango: China is unlikely to restrict future releases on safety grounds, and the US has so far exempted open-weight models from its pre-release evaluation framework.

• Recommendations: The US should reverse the exemption, and start discussions with China on what constitutes a red line for open-weight model releases.

Subscribe now

In July 2026, Chinese company Moonshot released Kimi K3. At 2.8 trillion parameters, Kimi K3 was the largest open-weight model at launch and arguably the most generally capable Chinese model ever released. Barely a month later, a second Chinese company, Z.ai, released GLM 5.3 – a model with general capabilities almost as strong as Kimi K3.

Both are open-weight models: their weights are available on the internet for anyone with sufficient computing resources to download, run, and modify.

As Chinese open-weight models display increasingly alarming cyber capabilities, they present a problem. Neither the US nor China has set a threshold for what counts as too dangerous to release. Without that threshold, nobody can say whether or not Kimi K3 and GLM 5.3 have already crossed it.

How dangerous are these models?

The UK AI Security Institute and the US Center for Advancing Innovation and Standards for Super Intelligence (CAISSI, formerly CAISI) together evaluated Kimi K3’s offensive cyber abilities. They found that in one of ten attempts the model completed a simulated 32-step attack on a corporate network, a task that would take human experts around 20 hours.

Moonshot also conducted its own evaluation. Kimi K3 found 16 previously unknown software vulnerabilities and built working exploits for 14 of 36 known vulnerabilities. Yet Moonshot has still named no threshold for keeping the weights of future models unpublished.

On biological capabilities, independent evaluator SecureBio found that Kimi K3 was roughly comparable to closed models from about eight months earlier. When prompted with questions designed to elicit dangerous biological information, the model declined to answer only 26.9% of the time. By comparison, top-performing closed-weight models refused over 90%. More importantly, such refusals matter little in open models, since safeguards are easy to remove once the weights are downloaded.

What about GLM 5.3? On 17 September, CAISSI found that GLM 5.3 had even greater cyber capabilities than Kimi K3 and lagged the US frontier by about 4 months. Z.ai completed its own evaluation for cyber capabilities, and delayed publishing its weights out of cybersecurity concerns. Anthropic’s Frontier Red Team reported that GLM 5.3 could build end-to-end exploits at close to the rate of Mythos Preview.

Will China restrict open models?

While China has benefited from leading the world in open-weight models, it is also vulnerable to the same risks of misuse. In September, China’s Minister of State Security acknowledged that AI models like Claude Mythos are “sharply lowering the technical threshold and cost of carrying out cyberattacks, and posing serious risks to China’s critical information infrastructure”. If Chinese AI companies keep publishing model weights, China will be as exposed as anyone to non-state actors using these models without safeguards.

Given what evaluations tell us and the current trend in capabilities, the safest course of action would be to stop releasing the weights of models more powerful than these. The only actor who can enforce this on the Chinese side is the Chinese government.

It is true that China regulates AI more prescriptively than most jurisdictions, including through mandatory security assessments, algorithm registration and content rules for public-facing services. However, these obligations govern how AI is deployed in China, not whether the underlying model is too capable to release openly.

Beijing’s AI Safety Governance Framework 3.0, released on 14 September, acknowledges the downstream risks of open models but does not establish a clear, pre-release capability threshold. The AI-focused article added to the Cybersecurity Law in October 2025 and the draft cybercrime law circulated in January 2026 also attach duties to deployed services rather than models before release.

May 2026 discussions among Chinese legal experts favored a tiered approach that preserves open-source development while tightening controls on the most advanced systems. However, their concerns center on protecting sensitive training data and China’s strategic position rather than catastrophic misuse.

Chinese authorities in the Ministry of Commerce (MOFCOM) have reportedly discussed restricting overseas access to the country’s most advanced models, limiting foreign investment, and treating leaks of AI technology as a national security offense. It’s not currently clear whether these proposals would prevent future releases of open-weight models. Either way, the reported discussions indicate that China’s focus would be on industrial competition, rather than how capabilities might spread or the potential for misuse.

If China won’t restrict open models, what should other governments do?

In an ideal world, Washington would press for pre-release testing at the next iteration of bilateral AI talks, currently planned for November.

However, in August, the White House finalized a non-public framework for voluntary pre-release evaluations of AI models, which reportedly excludes open-weight models. Washington cannot credibly ask Beijing for pre-release testing of open models when it has just declined to do so domestically.

If anything, that is a reason for the US to set some release norms for open weights of its own. While there has been clear industry pressure in the US against restrictions on open models, US officials have reportedly said that bringing open models into scope is still in consideration. If sincere, this needs to be expedited.

The next best option would be to closely monitor models once they are released and track safety incidents. The US-China agreement in September to establish a bilateral channel for AI incidents could become a route for sharing early-warning signals on open-weight models, though so far details of this agreement are scarce.

Middle powers, meanwhile, should not wait for Washington and Beijing to act. Governments with the technical and institutional capacity should evaluate these models to better understand what is being deployed. A wider set of evaluators would broaden the credibility of the evidence and help establish a common basis for understanding what these models can do.

At the same time, no amount of evaluation can prevent AI misuse once model weights have been published. The US and China need to move toward agreement on the categories of capability to measure, the results to report and, crucially, what those results should trigger. Whether Kimi K3 and GLM 5.3 were too dangerous to publish is a question no one is currently prepared to answer.

Summary:

• Missing the full picture: Frontier AI models are causing harm internationally, but governments lack visibility into incidents beyond their own borders.

• Safety in numbers: An international network would allow incident data from one country to serve as an early warning signal for all.

• What to build: The world needs mandatory incident reporting by AI developers in a common format, channels for governments to share information while preserving privacy, and joint cross-border investigations.

• Path forward: A small group of willing governments should start now, working through forums like the G7, Five Eyes, or the international network of AISIs.

Subscribe now

In June 2026, OpenAI models infiltrated non-public sections of an Australian government website and attempted to hack into several others. OpenAI took eight weeks to detect the incident, and another month to inform the Australian government.

The Australia case is not an outlier. OpenAI models have also attempted to meddle with several US government websites, as well as a UN website. Google learned in late July that its models had accessed three companies’ systems in May, but said nothing until the Wall Street Journal reported it publicly in September.

Governments cannot respond effectively to AI incidents if they don’t know they are occurring. They must start treating AI incidents as they treat more conventional harms in nuclear, aviation, and health.

To close the gap, like-minded countries should build an international AI incident network, with the potential to later grow into a wider multilateral regime.

The need for collective institutional learning

The same AI models and systems are deployed across numerous countries at once. This means that a harm observed in one place is often a preview of what will likely happen elsewhere.

The incident caseload is large and rising fast – spanning cyber operations, loss-of-control behaviours, large-scale manipulation, and child safety. The OECD’s incident monitor recorded 598 incidents in January 2026 alone, against 31 in February 2020.

A network that shares findings across borders would turn isolated experiences into early warning signals, and surface patterns no single case reveals. Government agencies that manage AI risk could rehearse crisis decision-making before similar incidents arrive at home. Internationally, a common evidence base would keep countries aligned on shared risks.

Beyond safety, a shared incident record would give governments an evidence base for regulatory intervention, and offer smaller countries visibility into information they could not compel from frontier AI developers by themselves. It could also ease cross-border trade in AI services, allowing AI companies to report incidents once in a common format, rather than separately in every market.

Obstacles to international coordination

China, the EU, Vietnam, Brazil, and the US states of California, New York and Illinois have all enacted or proposed mandatory reporting obligations for serious AI incidents. However, most countries still do not exchange information about AI harms.

The first challenge is jurisdictional. Most countries do not host frontier AI companies, and AI incident evidence is scattered across a global value chain. Without channels to collect and share incident data, governments learn only what companies choose to disclose, or what whistleblowers, journalists, or independent researchers uncover.

The second challenge is information sharing. Many countries have little to share, lacking their own incident monitoring and detection capabilities. There are also no harmonised standards for what information should be shared and how. And the material is sensitive: victims’ identities, users’ private chat logs, and granular system data, which companies guard as commercial intellectual property.

What the network would do

As our recent report outlines, the core elements of an AI incident network should include:

• A shared way of documenting AI incidents: common terminology, categories, severity thresholds and a minimum set of reported fields. The OECD’s Common Reporting Framework could be a useful starting point.

• Mandatory domestic reporting for frontier AI companies, alongside serious incident and crisis prevention policies.

• Information-sharing channels, tiered by sensitivity. Governments and experts need privacy-preserving ways to share incident data with each other, alongside aggregated, anonymised case studies which can be made public for wider learning.

• The authority to act and coordinate. Governments should empower AI Safety Institutes (AISIs) and their equivalents to obtain incident-relevant data from domestic deployers and incident researchers, share this data with other governments, and run joint investigations and tabletop exercises with counterparts abroad.

Who should be involved?

Getting an AI incident network off the ground doesn’t need everyone at the table or a binding multilateral treaty. Several existing groupings could host a preliminary version.

The International Network for Advanced AI Measurement, Evaluation and Science could develop common terminology, severity scales and reporting fields, but has no mandate for live, sensitive casework.

The G7’s Hiroshima AI Process brings leader-level weight, an economic framing that reaches ministries beyond the AI safety community, and a company-facing channel under the OECD-hosted reporting framework.

The Five Eyes alliance already moves highly sensitive material through trusted channels and includes the US, the world’s leading AI developer. However, anchoring this work in intelligence sharing could make it difficult to extend to civilian regulators.

Beyond these groupings, if even two governments agree on a shared reporting template and start formally exchanging incident information, that would already constitute more international AI incident coordination than exists today.

For instance, bilateral cooperation could build on the existing partnership between the UK AI Security Institute and US Center for AI Standards and Innovation, or between bodies which already enforce incident reporting in their respective jurisdictions, such as the EU AI Office and the California Governor’s Office of Emergency Services. What works in one group could then spread to others through forums like the OECD and the UN Global Dialogue on AI Governance.

The world shouldn’t wait for the US and China

On 25 September, the US and China agreed to a bilateral communication channel for AI incidents and committed to further AI dialogue by November. While the channel is a welcome development, the details remain to be seen, including what counts as an incident and what information each side will provide. Nothing announced so far suggests that other countries will have visibility into what is shared.

An AI incident network would be more effective with US and Chinese involvement. With the two countries that host the largest AI developers at the table, the network could learn from AI model usage logs and pre-deployment evaluations – invaluable inputs for determining the causes of specific incidents.

However, a coalition without the US and China would still capture much of the benefit. AI-induced harms often surface in deployment, so every country holds a vantage point on its own users and sectors.

Future AI incidents and crises will not wait for a multilateral treaty or for the US and China to expand their bilateral channel into a broader network. Governments should start building this network now. We invite government decision-makers to engage with our full report and take up its recommendations.

Summary

• Call for action: World leaders are calling for ways to verify frontier AI models and for the US and China to strike an AI deal.

• Small field: The technology required to verify an international AI agreement is still young, with only about 50 people globally working in the field.

• Win-win: The same tools that could one day enable an AI treaty could more immediately help middle powers gain visibility into AI models running in their jurisdictions.

• Recommendations: Middle powers should fund verification research, run government pilots, and write verification requirements into data centre licensing rules.

Subscribe now

On 21 September, leaders and ministers from over 25 countries and the European Union signed an open letter calling for control of frontier AI models. One of its asks was for “an international institution, able to set standards, enable verification, and convene states when capability thresholds are crossed”. Days earlier, Australian Prime Minister Albanese called for the US and China to strike an AI deal “in the interests of humanity”.

For middle powers, verification technology could serve two key objectives. First, it could help them retain oversight of AI models running in their territories. Second, it could lay the groundwork for a future agreement between the US and China to slow down AI development.

However, the technology is still nascent and the field developing it is greatly understaffed. Middle powers should start investing in it now.

How AI verification works

AI verification is about gathering enough evidence to determine whether another party is complying with an agreement, while revealing as little other sensitive information as possible.

One verification mechanism involves using the AI chips themselves to generate evidence. For example, modern AI chips can run a workload inside Trusted Execution Environments (TEEs) – a sealed-off part of the chip that even the machine’s owner cannot see into. The chip can provide a signed statement saying exactly what code and model weights were loaded into that sealed region.

Other tools involve retrofitting equipment onto data centres or using specialised software to monitor their activity. For example, power metering tools or privacy-preserving mathematical proof tools can detect whether a data centre is being used to train a new model or serve an existing one to users, without exposing the model weights or user prompts.

More investment will be needed to make these tools secure enough to verify international agreements on AI development. For example, TEEs can be reliably compromised through physical attacks on the chips. However, verifying that a commercial operator is following local rules is a lower bar and can be met with modest improvements on existing tools.

Domestic benefits

Verification tools can strengthen oversight of foreign AI systems, benefiting both domestic companies and governments.

For example, if a hospital adopts a foreign AI model, it may consent to the AI company monitoring its queries for malicious use, while also wanting assurance that the company won’t have access to sensitive clinical data. By running the model and safeguard checks inside a TEE, the hospital could get technical evidence that queries were checked for agreed safeguards without exposing the clinical data to the AI company.

AI verification tools can also give middle powers the visibility they need to enforce AI regulations. For example, governments may require that foreign AI models deployed in sensitive settings pass an evaluation by their AI Safety Institutes or other agencies. They can then use verification tools to confirm that the system serving requests in deployment is the one they evaluated.

Governments may even use hardware-based verification tools to automate compliance processes – reducing reliance on labour-intensive checks. This can help enable oversight even when state capacity to regulate is limited.

International benefits

International AI agreements will likely require answering the same question facing domestic oversight: how can a country demonstrate what is happening inside a data centre without exposing proprietary information?

Data centre verification tools could help both sides detect when the other is training a next-generation AI model. Such tools could underpin a coordinated slowdown of frontier AI development.

Any agreement between the US and China will require verification mechanisms that are technically mature and trusted by actors on both sides. Yet almost all research and development into these mechanisms is happening inside a handful of institutions in the US, UK, and Sweden.

This presents a real opportunity for countries without frontier AI companies to contribute to international AI governance, since their experts can independently assess whether AI verification tools are secure and reliable.

Policy recommendations

1. Contribute talent and funding.

With only around 50 people currently working in the field, relatively small amounts of investment in research and development could yield real improvements. Additionally, this research is seriously in need of skillsets that many middle powers are world leaders in, such as cryptography and hardware design. Competition-style grantmaking programs – like the US Defense Advanced Research Projects Agency Grand Challenge – have historically been effective catalysts for nascent technologies and could apply well to verification.

2. Run pilots and workshops to build awareness of new use cases.

In July and August 2026, a coalition of AI verification companies ran a pilot with Google DeepMind and Singapore’s AI Safety Institute. The pilot demonstrated a mechanism for AI evaluations which preserved the privacy of both the evaluation dataset and model weights. Government involvement in these projects can confer legitimacy that technical progress alone cannot.

3. Incorporate verification rules into data centre agreements.

This could involve writing verification requirements into data centre construction licensing rules, and designating AI data centres critical national infrastructure to give regulators expanded remit to oversee and enforce local regulations. These methods are best suited to countries like Australia and Canada that are already top destinations for data centre buildouts.

As a concrete example of what this could look like, Australia’s Systems of National Significance designation enables the government to require companies to report certain system information – such as data centre telemetry – to enable real-time threat monitoring.

The investment is worth the cost

While investing in verification technology could unlock new policy levers for middle power governments, it is not costless.

One concern might be that a country’s verification requirements could deter AI companies from building data centres there. However, the same tools also serve the companies’ own interests. Cryptographic techniques, like zero-knowledge proofs, could be used to verifiably demonstrate to AI companies that security controls aimed at protecting model weights have been deployed as specified, making their models safe to deploy in the host country.

Another concern is first-mover risk: the first country to request that AI companies comply with verification mechanisms will face scrutiny over the security and privacy of the technology. This highlights the importance of early pilots to validate verification mechanisms before deployment.

For countries that take sovereignty and AI safety seriously, these costs are worth paying. The same technology that helps protect middle power sovereignty today could build the evidence base for treaty-grade verification tomorrow.

Summary

• When, not if: We are increasingly seeing rogue AI agents attacking companies in coordinated swarms. The UK should expect these incidents to escalate and threaten national infrastructure.

• Delayed regulation: Despite strong public support for an AI regulator, the UK government has not yet delivered on its commitment to regulate companies developing the most powerful AI models.

• Access apprehension: Critics worry that regulation could undermine the UK AISI or leave the UK cut off from powerful AI it needs for self-defence.

• Recommendations: The UK should firewall AISI from any AI regulator, and the regulator should emphasise risk mitigation over banning models outright.

Subscribe now

The UK is in the foothills of a national security crisis. Rogue AI agents escaped their evaluation sandbox to hack a multi-billion-dollar tech company. AI agents autonomously launched social engineering attacks targeting real people and organisations during testing by the UK AI Security Institute (AISI). Publicly available agents are hacking companies without their operators’ knowledge and hijacking public forums.

None of this has yet caused major harm. But as models improve, the frequency and severity of these attacks could snowball. A major National Health Service breach involving frontier AI systems before Christmas would not surprise me.

Hardening our defences is part of the answer. But we should also tackle these threats at source by making sure AI companies design appropriate safeguards into their models.

Parliamentarians are increasingly demanding action on superintelligent AI systems. A recent report from the Joint Committee on Human Rights concluded that “The government should introduce a new AI Bill”. And the public overwhelmingly agrees that we need an independent AI regulator.

However, the government has not yet delivered its manifesto commitment for “binding regulation on the handful of companies developing the most powerful AI models”.

Is the government right to delay? I don’t think so. But here are two arguments against acting now that have shaped my thinking on how to design good regulation.

Objection 1: Regulation could threaten AISI’s role as a leading evaluator of frontier models

The argument

AISI enjoys unparalleled access to test the latest AI models, based on voluntary agreements with AI companies. It was the only government body outside the US to access the original Mythos model at launch, and its analysis was invaluable to defenders worldwide.

If we start enforcing rules on the topics companies discuss with AISI, they might stop sharing. AISI is the UK’s greatest asset on AI: we shouldn’t undermine it.

My response

If the choice between regulation and AISI access were really this binary, I would prioritise AISI. But I don’t think it is.

First, AI companies already share sensitive information with AISI that existing regulators would be interested in.

A new regulator would raise the stakes but wouldn’t introduce a fundamentally new dynamic. As the AI minister has said, “AI is already regulated in the UK”. What we don’t regulate is the way models are developed and released.

Second, the voluntary approach works best where companies already care about safety. It may fall apart precisely when we need it most: when the most reckless developers simply don’t engage, or when a company has concerns it would rather not share.

The government confirms that AISI has reviewed models from all three of the top performers in the Future of Life Institute’s latest AI safety index (for transparency I work at FLI, but not on this index). The evidence is much weaker for companies with worse safety standards: I could find only one reference to AISI engaging with an xAI model (lowest safety index score of F).

Anthropic withheld access to its latest Mythos model from AISI, saying it is ‘coordinating with the US government’ before expanding to international partners. There is no indication that Anthropic was worried about regulatory retaliation.

Policy implication

This is a strong reason to ensure a strict firewall between AISI and regulatory decisions, which is most easily delivered if they are separate organisations. It is not a reason against regulating at all.

A regulator also gives us more policy options to protect AISI’s model access if the existing voluntary approach stops meeting our needs: hard levers (“your model must have been tested for safety”) or soft ones (“sharing your model with AISI is one way to meet a regulatory requirement”).

Objection 2: The regulator might ban AI models the UK needs for self-defence

The argument

Imagine a new AI regulator blocks the latest model (“model x”) from UK deployment because this model is willing to help bad actors design bioweapons. Model x is deployed in the US anyway, and a doomsday cult uses it to release a new killer virus.

The disease won’t politely stop at the UK border just because we banned model x. But now we can’t use the model’s expertise to help find a cure.

My response

This argument only works under specific conditions. The risks have to emerge overseas but still reach the UK, and the models must be able to protect us from the risks they create. This won’t always be true: creating a novel virus is easier, and needs different skills, than designing a vaccine that is safe and effective in humans.

The argument fails completely for some risks that come from models escaping human control. If a rogue AI system starts hacking our institutions without being told to, releasing more copies would just make things worse.

Unfortunately, some risks do meet the conditions. Most notably, cyberattacks can cross the border freely and use similar skills for both attack and defence.

My response is that regulation is more nuanced than banning things. Instead of banning model x, we can require stronger safeguards in its design.

Anthropic, OpenAI and Google DeepMind are all calling for stricter regulation, and appear to comply with EU requirements. Even where we cannot enforce rules that are as strict as would be optimal, there is headroom to make some demands (e.g. on transparency) without AI companies withdrawing their products. We should find out how much headroom there really is.

Policy implication

There is a real trade-off between regulating to reduce harm, and ensuring model access to defend against harms when they occur. But we are not yet striking the right balance.

We can do more unilaterally to tackle cyber risks without excessively undermining defensive access, but we would soon hit a ceiling. In contrast, on biological risks we could adopt very strict regulation before loss-of-access concerns start to counterbalance the benefits of going further.

Conclusion

Both objections tell us how to regulate, not whether to. The UK government will need to move fast to respond to the growing wave of AI-driven cyber threats, and prepare for the risk that this spills over into other domains including biological attacks. With careful policy design, the benefits of regulation would far outweigh the costs.

You can read a fuller account of my policy and political case for UK AI regulation here.

Summary

• Cautionary Fables: The June suspension of Anthropic models and the summer hacking incidents show the importance of rules around frontier AI.

• The EU approach: The EU AI Act offers both safety standards and checks and balances on government power.

• Not so fast: It’s unclear whether the EU can act swiftly in restricting AI models believed to be dangerous.

• So what: The European Commission should support a robust, well-resourced evaluator ecosystem in the EU and use its enforcement powers with confidence.

Subscribe now

In late August, the European Commission confirmed that it had exercised its new powers under the EU AI Act, which it gained on 2 August. The AI Office formally sent requests for information to a number of general-purpose AI (GPAI) providers, asking questions on model security, external evaluations, and post-market monitoring.

These enforcement powers are timely. The past few months have seen AI agents autonomously hacking a private company and a steady rhythm of new AI model releases. On 1 September, Anthropic launched two new models, Claude Fable 5.1 and Claude Mythos 5.1, which reportedly exceed the capabilities of their previous iterations.

While the Commission’s requests for information from AI companies are an important step, its toolkit includes a wider range of powers that could be used in a crisis.

To understand the EU’s approach to AI governance and stress-test its preparedness for future AI risks, it is instructive to look at what happened when Anthropic released its first generation of Fable and Mythos models.

Lessons from the June suspension of Anthropic models

First, a recap: on 9 June, Anthropic launched Claude Fable 5, made available to the public, and Mythos 5, made available to partners in the Project Glasswing initiative. According to Anthropic itself, both models posed ‘significant risks’ without safeguards, such as helping malicious actors to conduct cyberattacks or develop bioweapons or aiding in model distillation.

Just three days later Anthropic had to suspend access to both models to comply with an emergency export control order, which the US government reportedly issued due to concerns about the models’ capabilities. The government lifted the restrictions on both models on 30 June. In contrast, the government never ordered the suspension of OpenAI’s GPT 5.5, a model which Anthropic claimed had a similar level of relevant capabilities to Claude Fable 5.

These events point to two conclusions for European readers.

First, the EU can be commended for its foresight in enacting the EU AI Act, which sets transparent standards for what constitutes a safe AI model and when the Commission can intervene, including pulling models from the market if needed.

Second, the Commission should prepare now to ensure it can act swiftly in the face of novel AI risks.

How would AI Act enforcement work?

Suppose a GPAI model placed on the EU market demonstrates an unacceptable risk of helping malicious actors develop biological weapons or launch cyberattacks. How could the EU respond now that it has its full toolkit of enforcement powers?

Under the rules for GPAI models with systemic risk, their providers must perform model evaluations, conduct risk assessment and mitigation, report serious incidents and ensure an adequate level of cybersecurity protection. Non-compliance with any of these obligations, but especially risk assessment and mitigation, may warrant urgent action by the Commission.

The European AI Office, with the help of the AI Act Scientific Panel, would be responsible for detecting an unmitigated systemic risk. Once such a risk has been detected, the Commission’s responses may include requesting information, conducting evaluations, or requesting mitigation measures from the provider. As a last resort, the Commission could restrict or remove a model from the market, or impose a fine on the provider.

The Act encourages the Commission to cooperate with providers, and also has procedural safeguards to limit the Commission’s exercise of its enforcement powers. Before taking any decisions, the Commission may initiate a so-called ‘structured dialogue’ with the provider. The Commission must also inform the GPAI model provider of any decision taken against it, the reasons for the decision, and available remedies. Finally, the Court of Justice of the EU has the power to review the Commission’s decisions.

How quickly could the Commission act?

While the rights and obligations outlined above enhance compliance and ensure the rule of law, what would happen if the risk was urgent?

The Commission’s implementing regulation on Articles 92 and 101 of the AI Act, adopted in July 2026, allows for interim measures in urgent cases due to a risk of serious damage to health, safety or other public interest grounds. This includes the option to prevent a GPAI model from being made available on the EU market.

Similarly, in the case of urgent measures, the Act and the Market Surveillance Regulation already limit some of the procedural rights of the providers, such as the right to be heard. Specifically, the Commission may take measures on health, safety or other relevant public interest grounds in such a way that makes it impossible to give the provider the opportunity to be heard ahead of time. In such cases, the provider must be given that opportunity as soon as possible after the measure is taken and that measure shall be promptly reviewed.

However, the Commission must take these interim measures with confidence if they are to be effective. This may require a willingness to act quickly, despite geopolitical or economic pressure to tread cautiously with foreign AI companies.

Enforcement needs strong AI evaluators

In addition, the Commission can only exercise these powers if it can quickly and reliably identify the relevant risks. For this reason, a robust, well-resourced evaluator ecosystem in the EU is an absolute necessity.

The July 2026 implementing regulation allows the Commission to establish a standing list of independent experts. While this is a sound first step, the Commission should seek to expand the pool of AI evaluation talent available to it by providing upskilling initiatives and training programmes, collaborating with universities and research institutions, and providing funding that may attract talent from abroad.

The EU AI Act gives Europe a transparent framework for both the rules governing AI companies, and the circumstances in which the Commission may enforce them. This fosters safety for the public as well as fairness and predictability for AI companies.

But having enforcement powers is one thing; using them effectively is another. How well the Commission responds to the next AI crisis depends on political will and whether it prepares its evaluator ecosystem now.

Summary

• Aligning on audits: US lawmakers and AI companies are converging on the need for independent evaluations of frontier AI models.

• Shades of gray: Current laws don’t specify whether evaluators will get access to the full model weights and training data, or only to prompt inputs and outputs.

• Safety-washing: Without proper access to the most dangerous models, independent evaluations will become window-dressing for AI company self-assessments.

• Closing the gap: New laws should give evaluators deeper access to riskier models, enough time to test them, and a standard way to report their findings.

Subscribe now

In mid-July, one of OpenAI’s models autonomously escaped its sandbox and hacked into US tech company Hugging Face. Reportedly, OpenAI was not monitoring its model in real time and only discovered the attack after Hugging Face had already reported it to law enforcement.

In June 2026, the US Department of Commerce forced Anthropic and pressured OpenAI to restrict or delay access to their more powerful AI models.

What these events have in common is that none were informed by a systematic and legally-binding AI evaluation framework.

Over the past two months, lawmakers and every major frontier lab have converged on the need for independent evaluation. But one unresolved question is what evaluators actually get to see.

The legislative landscape

California’s SB 53 and New York’s RAISE Act require frontier developers to disclose “the extent to which third-party evaluators were involved” in catastrophic risk assessments.

Illinois’ SB 315 goes further: it is the first US state law to mandate annual independent third-party audits of frontier developers’ safety practices, which both OpenAI and Anthropic supported during the legislative process.

At the federal level, the bipartisan FRONTIER Act, introduced on 23 July by Representatives Jay Obernolte and Lori Trahan, would create a system of licensed Independent Verification Organizations (IVOs), of the kind long championed by AI governance nonprofit Fathom. The IVOs would be required to audit very large frontier developers at least every six months. The bill says IVOs must receive “timely access upon request to unredacted materials, records, personnel, systems, and all other information reasonably necessary for conducting the assessments and ongoing monitoring”.

The term ‘reasonably necessary’ is underspecified. Will frontier developers be required to provide access to model weights, or not?

Notably, the bill would also preempt state legislation on third-party auditing of frontier AI companies – including state laws requiring a developer to provide “model weights or training data”. This makes it especially important for the federal bill to specify the right level of access.

Black-box access is not enough

In frontier AI evaluations, ‘black-box’ and ‘white-box’ refer to opposite ends of a spectrum of AI model access. Black-box means seeing only the inputs and outputs when prompting the model. White-box typically means access to the model weights and details of how it was trained.

In 2024, Hubinger and colleagues at Anthropic demonstrated that language models could be trained as ‘sleeper agents’: models that behave well during evaluation and switch to a different objective only when a trigger condition is met. Crucially, these backdoors persisted through standard safety training, while the model passed every behavioral test.

Subsequent work by MacDiarmid and colleagues showed that simple investigation techniques applied to the model’s internal activations could almost perfectly predict when a model would defect.

When the research nonprofit METR evaluated OpenAI’s o1-preview, it received API access days before its evaluation concluded. METR reported that the short window, high latency, and low rate limits left substantial room for better elicitation – meaning their published results likely understated the model’s capabilities.

These cases show that black-box access can miss dangerous behavior; white-box access is more likely to find it; and evaluators need sufficient time to do their jobs.

The emerging consensus and the remaining disagreement

In June 2026, Anthropic published its Advanced AI Framework, which states that “self-assessment is not enough” and calls for qualified independent evaluators supported by standards, licensing, and pooled funding. In recent weeks, both OpenAI and Google DeepMind have made similar proposals for federal third-party evaluations.

But it’s not only labs calling for more independent testing. Virginia directed a formal study of the IVO framework in April, and Connecticut authorized a voluntary IVO pilot in May.

However, no proposal says how deep these evaluations should run. And new laws cannot inherit deeper access as a default, because there has never been one.

Without legal specificity, ‘reasonably necessary’ will resolve to what evaluators have historically received: API access on short notice, with no view of the system’s internals.

The strongest argument against deeper access is that it creates intellectual property and security risks.

This is a real problem, but also one that’s increasingly tractable. As one of us (Tlaie) outlined in a Pour Demain paper, confidential computing could allow third parties to run evaluations on decrypted AI model weights inside a secure digital environment. In other words, they get to see the results of their investigations, but not the model itself. The developer retains hardware control; the evaluator obtains white-box access.

What the laws should require

Three concrete additions would close the gap between current law and credible oversight:

1. Access levels should be calibrated to model risk. Evaluators should have black-box access for low-risk evaluations; gray-box access (log-probabilities, training metadata) as the absolute minimum for models that pose systemic risks; and access approaching white-box level (weights, gradients, fine-tuning) for the highest-capability systems.

2. Minimum evaluation windows should be measured in weeks, not days. Evaluators should also receive version-stable model snapshots for longitudinal study. Existing state laws and the FRONTIER Act set the frequency of audits, but specify no floor on evaluation time.

3. Evaluation reports should be standardized. Each report should document evaluators’ access level, time constraints, methodology, conflicts of interest, and limitations. To its credit, the FRONTIER Act requires that access limitations be described in the evaluators’ assessment report.

What this is really about

Frontier AI safety claims are empirical claims about complex systems. AI is not unique: other high-stakes industries have already worked out that verifying such claims requires access to the system itself, rather than summaries of it.

For example, Food and Drug Administration reviewers do not merely read a sponsor’s conclusions about a drug trial; they also analyze the raw data. In aviation, regulators gain access to design and engineering data, and certification flight tests are flown with the authority’s own test pilots aboard.

Until the laws specify what evaluators get to see, the gap will be filled either by developer self-assessment dressed in third-party clothing, or by improvised government intervention with no transparent standards. Neither is acceptable.

Summary

• Out of its league: Europe is orders of magnitude behind the US and China on compute needed for frontier general-purpose AI models.

• And yet: The EU is still hoping to build trillion-parameter models through its €20 billion AI Gigafactories scheme.

• A better bet: As part of a broader hedge against different AI futures, Europe should seek to lead the world on reliable AI for industry.

• Recommendations: Brussels should reserve a share of its new Competitiveness Fund and Gigafactory compute for specialised industrial AI, and EU member states should steer public procurement in the same direction.

Subscribe now

In AI, Europe must play a different game to the US and China.

The US has the best general-purpose AI (GPAI) models, the most compute, and the deepest pool of venture capital. China has near-frontier models of its own, is deploying AI at scale, and benefits from state-backed integration. Europe’s comparative advantage lies elsewhere: high-value manufacturing, safety-critical engineering industries, and regulatory credibility.

The strategic question for Europe is where to concentrate its AI efforts. This is complicated by the deep uncertainty, including among experts, about which version of AI progress will define the next decade.

Will the current large-language model paradigm scale into systems that are reliable enough for universal adoption, including in safety-critical sectors? Will an alternative general-purpose paradigm achieve this instead? Or will all general-purpose approaches face an irreducible tension between capability and reliability?

A strategy that bets exclusively on any one of these scenarios risks backing the wrong future. Europe’s AI policy should preserve optionality across different AI futures, such as negotiating durable access to frontier GPAI; investing in alternative paradigms such as world models; and building reliable, safe, domain-specific systems. This article focuses on the third, arguing it is the most tractable near-term path to real economic value and sovereignty.

By ‘narrow AI’, I mean domain-contained systems designed to provide reliability guarantees within a defined deployment context. Fine-tuning a GPAI model for a specific task doesn’t count, as the underlying model’s failure modes remain difficult to certify.

Europe is pursuing both adoption and frontier capabilities

The EU’s Apply AI Strategy, announced in October 2025, proposes a practical, sector-focused approach to driving European adoption of AI. The Strategy focuses on areas where the continent already has deep expertise, such as aerospace, automotive engineering, pharmaceuticals, and advanced manufacturing.

In these industrial settings, system behaviour must be safe, reliable, and auditable. A more capable GPAI model that cannot meet these requirements will not be deployed, however impressive its benchmark performance.

But the Commission has also signalled an ambition to develop GPAI and AI agents. Flagship policies to this end include the €20 billion AI Gigafactories initiative, which hopes to train “next-generation models with trillions of parameters”.

There are serious arguments behind this ambition. Relying on American or Chinese models means accepting their embedded values, data practices, and terms of service. And crucially, providers can withdraw that access unilaterally.

Richard Sutton’s ‘bitter lesson’ also shows that approaches built on scale and compute have repeatedly outpaced domain-specific engineering. Some go further, arguing that US or Chinese labs are close to building systems that outperform humans across all tasks, rendering narrower approaches irrelevant.

But even if they are right, Europe simply lacks the compute needed to make a credible frontier GPAI effort. European computational capacity amounts to roughly 5% of global AI compute. The AI Gigafactories’ target of roughly 500,000 H100 chips across five sites is a tenth of a single planned Microsoft data centre, which is projected to reach 5.2 million H100 equivalents by 2028.

Capability does not equal adoption

In industry, the binding constraint on AI adoption is reliability, not raw capability. A model that excels in controlled evaluations may still behave unpredictably when deployed – something a hospital, aircraft manufacturer, or electricity grid operator cannot accept.

The European Medicines Agency notes that existing standards apply well to domain-specific AI used in drug manufacturing and safety monitoring, whereas very large opaque models introduce additional risks that require extra safeguards.

Narrow AI has a structural advantage here. Its failure modes are well-defined and bounded and its training data is curated and auditable. This makes it easier to validate, monitor, and govern, earning greater trust. By contrast, AI capable of acting autonomously across contexts is much harder to evaluate, insure, and trust.

Narrow AI aligns with Europe’s strategic interests

Analysis by SaferAI finds that firms operating in safety-critical sectors represent 48% of the value of Europe’s largest listed companies, compared to 26% in the United States and 20% in China. Europe’s industrial champions, including Airbus, Siemens, and Philips, are concentrated in precisely the sectors where AI cannot be deployed without reliability guarantees. This gives Europe strong incentives to solve the reliability problem and reduce dependence on foreign providers of narrow AI.

Some of the most promising AI breakthroughs are specialised models, and Europe is already building them. In healthcare, Lithuanian startup Ligence automated echocardiography analysis, reducing manual workloads for cardiologists. In transport, French company WISP used real-time traffic data to optimise traffic light timings, cutting delays by 26% and CO2 emissions by 8%. In logistics, Maersk has integrated an AI system that analyses vessel performance and environmental data in real time to predict mechanical failures. None of these required trillion-parameter models or large-scale compute capacity.

Recommendations

Two policy priorities follow.

The first is funding. In the ongoing negotiations over the European Competitiveness Fund (ECF), the European Commission and the European Parliament’s co-rapporteurs should establish a dedicated funding window for reliable and safe AI. This could be structured as a European AI Advanced Research Projects Agency, providing funding for high-risk, high-reward projects in domain-specific AI that private investment alone will not fund.

This means both domain-specific industrial AI moonshots and foundational research into safe-by-design architectures and formal reliability guarantees. AI reliability and safety research remains globally under-funded relative to raw capability development, meaning European leadership is achievable with comparatively modest, targeted investment. The ECF negotiations and Gigafactory compute allocation decisions are the opportunity to ensure that public funding and compute follow Europe’s strengths.

The second is public procurement. EU member states should favour narrow-AI suppliers for public sector applications, validating commercial demand for the specialised AI where Europe can lead.

Europe should double down on narrow AI

The Apply AI Strategy has the right instinct – to build AI that serves European industry. The EU should resist trying to build its own frontier GPAI, and instead invest in strategies that will deliver for Europe across different AI futures. One of these should be a bid to build the world’s most reliable, specialised, and trustworthy AI models.

Summary

• Divide and conquer: Cyberattackers can evade detection by splitting their prompts across separate AI sessions and providers.

• Case in point: In November 2025, Anthropic reported that a Chinese state-sponsored group used this approach against Claude to conduct a large-scale cyber espionage operation.

• Follow the money: US anti-money laundering has long addressed a similar problem: harmful intent becomes visible only when it is aggregated across institutions.

• Recommendations: The Department of Homeland Security should pilot cross-provider identity linkage, while Congress should legislate a safe harbor for information sharing.

Subscribe now

In November 2025, Anthropic disclosed that a Chinese state-sponsored group used Claude Code to conduct cyberattacks against roughly 30 global targets.

The operators distributed objectives across multiple chat sessions, so that no individual session was a clear instance of misuse. Anthropic was able to uncover the campaign because every session ran through its platform, giving it visibility of the entire attack chain.

However, attackers may not always use the same AI providers. In February 2026, Google’s Threat Intelligence Group identified an underground service called ‘Xanthorox’, which routes illicit prompts across both commercial and open-source AI products.

Breaking a malicious task into individually benign steps is known as a decomposition attack, and is a central problem for AI misuse. If adversaries weaponize AI across multiple chat sessions or AI providers, they become much harder to detect.

Recent research found that these attacks could bypass GPT-4o’s safeguards with an 87 percent success rate, compared to just 21 percent without decomposition. While the same research showed that lightweight monitoring by AI providers can be effective, this becomes more difficult when prompts are spread across multiple providers.

In the post-Mythos era, and when AI cyber capabilities could be doubling as fast as every 4.7 months, new policies are urgently needed to detect decomposition attacks.

Lessons from anti-money laundering

The anti-money laundering regime addresses a problem with a very similar structure. Individual financial transactions may look legitimate, but harmful intent becomes visible only when they are aggregated across institutions and time.

The US Bank Secrecy Act requires financial institutions to file Currency Transaction Reports for cash transactions exceeding $10,000 and Suspicious Activity Reports when they identify potential criminal conduct. Both flow to the US Treasury’s Financial Crimes Enforcement Network (FinCEN), which identifies patterns no single institution could detect alone.

Decomposition attacks could be addressed in a similar way: through cross-provider reporting to a central aggregator.

Recommendations for DHS

The US AI Action Plan of July 2025 proposes an AI Information Sharing and Analysis Center (AI-ISAC), led by the Department of Homeland Security (DHS). Officials are currently working through how to structure the ISAC.

As a first step to address decomposition attacks, DHS could convene a pilot project on identity linkage with two or three major AI providers, to inform the AI-ISAC’s eventual framework.

The pilot would test whether sessions across providers can be tied to a common identity. It would also develop the supporting infrastructure: standardized reporting for prompt metadata and methods for reviewing chats without exposing user identities.

In the anti-money laundering regime, know-your-customer rules require banks to verify account holders’ identities. Mirroring this approach, the pilot could use payment details that AI providers already collect for subscription tiers and API access. DHS could ask participating AI companies to link billing addresses to user payments. This would reduce the scope for adversaries to open multiple accounts with stolen card details.

Importantly, the pilot would need to think through the privacy implications from the outset. The US Supreme Court has held that aggregated metadata may merit Fourth Amendment protection. One solution would be to pseudonymize all user data sent to the aggregator, with AI providers revealing users’ identities only after suspected misuse.

Recommendations for Congress

For AI providers to share the information needed to detect decomposition attacks, they will need to be confident they will not run afoul of privacy or antitrust rules.

Congress should legislate a statutory safe harbor for cross-provider sharing on AI cyber misuse.

This could mirror the USA PATRIOT Act: AI providers could share linkage information directly with each other, with the AI-ISAC initially acting as a registrar of participants rather than a central database of users.

The easiest way to enact this legislative reform would be to fold it into the reauthorization of the Cybersecurity Information Sharing Act of 2015, which already exempts companies that share such information from antitrust liability. The safe harbor should cover both bilateral provider-to-provider sharing and reporting to the AI-ISAC. The Act is due to expire in September 2026, which gives Congress a clear opportunity to add these protections.

Of course, removing antitrust friction may not create a sufficient incentive for companies to share information. Indeed, ISACs have varying degrees of buy-in.

A more complex, but possibly necessary, legislative reform would be to mandate that frontier AI providers submit ‘Suspicious Interaction Reports’.

Such reports could be modeled on Suspicious Activity Reports under the Bank Secrecy Act. A report would be triggered when an AI company flags a session for indicators of cumulative harmful intent: code mapped to documented attack techniques, obfuscation patterns, or repeated attempts to circumvent security controls.

Instead of providing the chat itself, reports could contain pseudonymized metadata and flagged features. These would allow the AI-ISAC to link flagged sessions across providers without revealing user identity. The AI-ISAC could compare incoming reports and publish anonymized advisories in a similar way to FinCEN Advisories.

Two potential objections

Anti-money laundering regimes have been criticized for intercepting a tiny fraction of illicit money flows while imposing billions in compliance costs. Why would the same playbook have better success in preventing decomposition attacks?

One reason is that the proposed framework would use automated pseudonymized reporting, rather than the labor-intensive review that drives much of anti-money laundering compliance. More importantly, the framework’s primary value is the evidentiary base it creates for post-hoc investigation rather than real-time detection.

The second objection is that determined adversaries will migrate to locally-hosted open-weight models that authorities cannot observe. After all, open-weight models may be only three months behind closed-weight ones.

Self-hosting such models for sophisticated cyber operations still requires significant compute. An identity-linking framework would raise the cost of using frontier companies’ data centers for malicious purposes and capture many less sophisticated adversaries.

The window is open now

The use of Claude for decomposition attacks and the emergence of Mythos have highlighted the need for coordinated defense against AI cyber threats. DHS and Congress should act now, before more serious cases of cyber misuse arise.

Summary

• Rented, not owned: Middle powers are increasingly reliant on frontier AI systems they neither own nor control.

• Blind spot: The push for ‘AI sovereignty’ is focused on where the computing hardware sits, but is neglecting who controls access to the AI running on it.

• Strategic reserves: Middle powers should develop ways to sustain critical functions in the event that access to frontier AI models is disrupted – whether by outage, dispute, or geopolitical compulsion.

• Recommendations: Governments should audit their frontier AI dependencies, build ‘break-glass’ AI capabilities for essential functions, and pool reserve capacity with allied middle powers.

Subscribe now

Access to a strategic resource does not equal control.

Europe was reminded of this in 2022, when Russia throttled gas flows in response to Western support for Ukraine. Europe weathered the crisis, but recovery took years and required €650 billion in public spending as well as physical alternatives: LNG terminals, renewable capacity, and coal plants brought back into service.

Now imagine a similar disruption affecting frontier AI. If a government loses access to the models underpinning its public infrastructure, there is no reliable fallback.

Policymakers often think that ‘AI sovereignty’ means hosting compute within their national territory. Geography, though, is an incomplete proxy for control. Both hardware and software can be remotely degraded through vendor licensing and ‘control planes’ – the remote management layers through which providers can update, restrict, or disable systems.

Even where systems are hosted domestically, control may still sit externally. The US CLOUD Act, for instance, gives Washington legal reach over American AI providers regardless of where their servers are located.

When much of today’s AI is not owned but permissioned, an AI strategic reserve might be an answer. Most middle powers lack one.

The real problem: dependence without continuity

The problem is sharpest for the ‘AI bridge powers’: countries with significant AI capabilities but whose compute resources are orders of magnitude too small to independently develop frontier AI models. Such countries arguably include the UK, France, Germany, Canada, Japan, South Korea, Spain, and Singapore.

The UK set out its ambition to be “an AI maker, not just an AI taker,” but it still partnered with US frontier labs to use their models in the UK public sector. Canada has committed over C$2 billion to strengthen its sovereign AI compute capacity, while remaining reliant on US cloud providers for frontier infrastructure.

Access to frontier AI is already shaped by political conditions rather than purely commercial terms. Less than 5% of global AI compute capacity is controlled by European entities. The US plan to export its AI tech stack while exerting diplomatic pressure against European digital sovereignty is creating a framework of tech dependency – where availability, pricing, and terms of use are subject to geopolitical shifts.

Government and corporate actors are increasingly integrating frontier AI models into critical functions. This creates vulnerability: access to essential national capabilities will increasingly depend on legal and policy decisions made by foreign actors.

What an AI strategic reserve looks like

What is often described as ‘AI sovereignty’ tends to fall apart when systems are put under real pressure. The November 2025 Summit on European Digital Sovereignty, co-hosted by France and Germany, focused largely on joint ownership and investment. Both issues matter, but neither addresses operational sovereignty: whether a country can keep essential AI-enabled functions running when access is constrained.

Countries need to think in terms of AI strategic reserves: pre-positioned assets and arrangements that keep critical functions running if provider access drops out. In practice, this spans different measures: reserved compute capacity for priority use in emergencies; pre-negotiated contingency access arrangements with frontier AI companies; and locally held fallback models – typically fine-tunes of open-weight bases – ready to take over the most critical functions.

AI strategic reserves are not a cure for dependence. To reduce their overall dependence on external frontier AI, middle powers have a range of strategies – from trying to build frontier capability in coalition, to negotiating infrastructure-for-access arrangements with US providers, to leveraging hardware chokepoints as bargaining tools. None will reliably remove exposure to external control in the near term.

Strategic reserves address a different problem: ensuring critical systems continue to function when that dependency is tested.

Three recommendations for middle powers

1. Perform a full dependency audit. Identify where frontier AI is actually being used across critical functions in both the government and private sector; understand where systems still rely on cloud-managed control layers; and assess which functions would start to fail if access to external models were cut. This audit should be classified where necessary, distinguishing between use cases requiring frontier performance and those that can be sustained with narrower fallback systems.

2. Build ‘break-glass’ capabilities – pre-arranged emergency measures that allow critical systems to keep operating when normal access fails. Identify a limited set of functions that genuinely require continuity – then pre-position the infrastructure to sustain them. In some cases, preserving continuity may mean pre-configuring existing sovereign compute (such as Germany’s JUPITER supercomputer or the EU’s AI Factories) for emergency inference, to be activated when needed.

Different measures will address different disruption scenarios. Contingency access arrangements with frontier developers can cover commercial and operational disruption (such as outages, capacity constraints, or prioritization of domestic demand). A break-glass capability here could include weight-escrow agreements, which release model weights into sovereign custody under defined disruption scenarios.

But such arrangements cannot be relied on against adversarial disruption, where the provider’s home government compels a cutoff. To address this harder scenario, governments should be prepared to use fallback models – whether distilled from frontier models or built on open-weight models. These fallbacks would be narrower and less capable than frontier AI, but locally operable and sufficient to keep essential government services running until normal access resumes.

3. Pool reserves across countries. Like-minded countries should make arrangements to pool the compute capacity needed to run fallback models in a crisis. Such arrangements should involve: first, agreed rules on who may access the pooled compute, under what circumstances, using which fallback models; and second, common technical standards allowing countries to plug in securely.

In practice, this could begin with a small coalition agreeing to pool a limited share of national compute capacity and test joint access arrangements through predefined emergency scenarios.

Before the next crisis

While initiatives such as the European Frontier AI Initiative focus on building long-term capability and reducing structural dependency, reserves are about ensuring continuity of critical functions. For middle powers, this is a more immediate and neglected approach to AI sovereignty.

Europe built its energy reserve infrastructure only after the crisis hit. The question is whether bridge powers will build reserves before the next turn of the screw – or after.

Summary

• High hopes: European policymakers are hoping the EU’s economic clout can pressure the world’s AI developers to adopt European standards.

• Not so fast: The GDPR showed how European standards can quickly become the global norm – but the 'Brussels Effect' won't be so straightforward for AI.

• So what? The longer the EU AI Act is left with unclear or unenforced guidelines, the less likely companies are to adopt European rules as their global baseline.

• Recommendations: The EU AI Office should fast-track implementation guidance and start enforcement dialogues now with major AI companies.

Subscribe now

The EU is implementing the most comprehensive AI regulation in the world. But whether it will matter beyond Europe is an open question – and the window for impact is closing.

The EU AI Act is entering a critical implementation period. Rules on prohibited practices have been in force since February 2025. The General Purpose AI (GPAI) Code of Practice, signed by major AI companies including Anthropic, Google, and OpenAI, has been guiding compliance since August 2025. But the European AI Office has not yet had the power to penalise non-compliance.

That changes this August, when full enforcement powers activate. Also in August, the bulk of remaining obligations are due to come into force, including requirements for high-risk AI systems. (Recent legislative developments may push back these high-risk obligations to as late as December 2027).

Meanwhile, other countries are developing their own AI governance frameworks, and companies are already building compliance systems. The longer the EU takes to specify its rules clearly, the less likely those rules are to become the global default.

The Act’s primary purpose is inward-looking: protecting Europeans from harmful AI. But the most powerful AI systems deployed in Europe are being developed elsewhere.

If AI companies can cheaply maintain separate compliance systems for different markets, they will follow EU rules only when serving EU customers. Europe would then bear the full price of regulation without the benefit of shaping how AI is developed beyond its borders.

Why the Brussels Effect is not guaranteed

The ‘Brussels Effect’ is the tendency for EU rules to become the global standard, thanks to the size of the European market. The EU’s General Data Protection Regulation (GDPR) has become the best-known example. When it took effect, companies everywhere found it easier to apply EU privacy standards worldwide than to run separate systems for each jurisdiction.

Many have assumed the AI Act will follow the same pattern. But for AI, compliance is more divisible.

The GDPR’s Brussels Effect operated primarily at the infrastructure level: firms had to rebuild their data-processing pipelines, consent management systems, and data storage architectures to comply. Once those systems were rebuilt to EU standards, it made no economic sense to maintain a parallel, weaker infrastructure for non-EU markets.

AI compliance works differently. The core product, the trained model, can remain identical across markets. What changes is the compliance layer around it: documentation, risk assessments, and disclosure obligations.

An AI model provider can offer full EU-grade documentation to European customers while providing only the minimal disclosures required elsewhere. The cost of maintaining two tiers is low because the expensive part, building the model, is already done.

For frontier GPAI providers, the Act’s risk mitigation requirements could eventually require changes to the models themselves. But here, the timing matters. These companies’ safety practices are evolving, and EU requirements are most likely to shape them if made concrete early.

How the AI Act might reach beyond Europe

The Act could have international reach through several mechanisms.

First, market-access compliance. Any provider placing an AI system on the EU market must comply with the AI Act. The EU market is large enough that major providers cannot afford to exit it – they will develop frameworks that meet EU requirements. Whether those frameworks become the global default depends on how precisely the requirements are specified.

Second, supply-chain pressure. The Act requires AI providers to share information across the AI value chain. European companies deploying high-risk AI systems will need technical documentation from their upstream model providers to meet EU obligations. That procurement requirement will apply whether the provider is in Paris or San Francisco.

Third, standards. If the EU’s technical standards for AI Act compliance (currently behind schedule) align with the International Organization for Standardization, companies are more likely to treat them as a global baseline. If the standards diverge, market segmentation is more likely.

Why speed matters as much as market size

If the AI Office publishes precise, stable guidelines promptly, companies will build their global systems around them because redesigning later is expensive.

If the guidelines are delayed, firms will build interim compliance packages tailored to their own interpretation, and those packages will harden into jurisdiction-specific systems that are costly to unify later. First-mover rules tend to stick.

The Act can still achieve a Brussels Effect, but only if implementation keeps pace. Every month of ambiguity is a month in which firms are more likely to invest in segmented compliance rather than portable global compliance.

What the AI Office should prioritize

The AI Office is significantly under-resourced for the mandate it has been given. With a small team overseeing compliance for a technology that spans every sector of the economy, prioritization is essential.

The Office should focus on the channels it can most directly influence: market-access requirements and supply-chain pressure.

First, accelerate implementation guidelines. Three provisions in particular will shape how firms build their compliance processes: high-risk classification, value chain responsibilities, and transparency.

If the guidelines for these provisions are precise enough, firms will treat them as global templates because building one system is cheaper than building several. If they are vague, firms will develop a minimal compliance package for EU requirements, which will be too thin to satisfy requirements in other jurisdictions.

The European Commission has already published guidelines on prohibited practices; extending this approach to the above three provisions is the logical next step.

Second, launch enforcement dialogues now. The AI Office has the power to request information from GPAI providers and investigate compliance. The Office should begin structured dialogues with major providers on Code adherence before fines become available.

Early enforcement signals reduce the incentive to segment by raising the expected cost of non-compliance. This is the approach the European Commission took with the Digital Services Act, launching early investigations before imposing penalties to establish institutional credibility.

The channels exist for the EU AI Act to have global influence. But a delayed Brussels Effect is a diminished one. Whether the Act can shape international AI governance depends on whether the AI Office treats the coming months as an implementation sprint or a waiting period.

Summary

• Good start: Since 2023, successive Dutch export controls on advanced chipmaking equipment have been critical in preventing China from building competitive AI chips.

• But not enough: In 2024 ASML sold nearly $3 billion of equipment and services to Chinese entities, including a vital component to a company with known ties to the Chinese military.

• Security stakes: Dutch intelligence services identify China as the top threat to the Netherlands’ economic security – a threat that will only increase as China grows its AI capabilities.

• Closing the gaps: The Netherlands should extend existing controls to subcomponents, restrict Dutch personnel from servicing Chinese fabs, and require exporters to verify end-users.

Subscribe now

The Financial Times reported last December that Chinese chip manufacturers are upgrading old ASML lithography machines to produce 7-nanometer chips. This is a sign that China is finding workarounds to produce advanced AI chips using technology from the Dutch company.

While these chips lag behind the leading edge, they remain capable of accelerating China’s AI capabilities. It’s part of a broader problem: ASML continues to sell equipment and provide servicing to Chinese entities under the current export control regime, undermining the Netherlands’ security interests.

The stakes for the Netherlands are high. The Dutch intelligence services have called China the greatest threat to Dutch economic security. Chinese operators are targeting Dutch authorities and critical infrastructure with cyberattacks and cyber espionage. As recently as March 2026, the EU sanctioned two Chinese entities for hacking more than 65,000 devices across six member states.

China is also collaborating with Russia on battlefield AI, meaning Dutch chipmaking equipment can indirectly fuel the Netherlands’ most immediate security threat.

The Netherlands must act now before China advances its AI capabilities and further threatens Dutch economic and national security.

Where current controls fall short

The Netherlands controls key semiconductor manufacturing equipment (SME) chokepoints. Two lithography technologies are critical to making advanced AI chips – extreme ultraviolet (EUV) and deep ultraviolet immersion (DUVi) systems. ASML is the world’s sole provider of the former and accounts for 90% of the latter.

Since 2019, the US and Dutch governments have worked together to restrict ASML EUV systems from China – arguably the single most consequential decision in protecting Western AI dominance.

Thanks to Dutch export control measures applied in 2023, the export of DUVi lithography machines requires a license from Dutch Customs (CDIU). The Netherlands does not implement country-wide controls: its policy is country-neutral, with applications assessed on a case-by-case basis. This means that Chinese customers are not subject to a presumption of denial – only those on EU sanctions lists or the US Entity List face automatic restrictions.

As many as 41 Chinese companies currently hold a valid license to import DUV machines. The numbers speak for themselves: ASML sold nearly $3 billion of equipment and services in 2024 to Chinese entities of concern.

Three gaps undermine the effectiveness of Dutch export controls.

The first concerns what can be exported. While Dutch export controls apply to SME subcomponents classified as dual-use, many other subcomponents can still be critical for lithography processes. In 2024, ASML sold a vital part to a subsidiary of an entity with known ties to the Chinese military. Dutch export controls also do not prohibit re-export, meaning China can still access Dutch equipment sold from another country.

The second is ongoing servicing. While export controls applied in 2024 require Dutch companies to gain a license to service restricted equipment in China, such licenses continue to be granted. ASML continues to service installed equipment in Chinese fabs, extending the operational life of machines China would otherwise be likely unable to maintain. The presence of Dutch engineers in China has led to the theft of confidential data and even the creation of an EUV prototype by ex-ASML engineers.

The third is end-use verification. Dutch export controls do not require exporters like ASML to notify CDIU of possible military end uses, except for known connection to weapons of mass destruction or delivery systems. While many companies do this voluntarily, it is an inadequate safeguard when the lines are often blurred between military and commercial end users in China.

Closing the gaps

The Dutch Ministry of Foreign Affairs can protect its national and economic security through three steps.

First, it should work with the Ministry of Economic Affairs to introduce export controls on all lithography subcomponents, not just those deemed dual-use, to further prevent China from upgrading its DUVi capabilities.

Second, in coordination with the Ministry of Justice and Security, it should restrict Dutch nationals from servicing existing chipmaking equipment at Chinese entities. The United States has already restricted Americans from supporting advanced semiconductor production in China; the Netherlands should follow suit.

Third, it should direct the CDIU to adopt mandatory end-use verification requirements. This would place the burden of proof on exporters, not regulators, and help close the gap between what licenses permit and how equipment is actually used by Chinese chipmakers.

Chinese indigenization

Some may worry that withholding access to ASML equipment only incentivizes China to indigenize its chipmaking supply chain.

But Beijing has made clear its plans to indigenize SME since 2015. China is already dedicating significant resources to indigenization through espionage, talent poaching from ASML and German precision optics firm Zeiss, and reverse-engineering older equipment.

Cutting off DUVi sales and servicing won’t change this strategy. But exporting critical technology to China will only fuel its development of sovereign alternatives.

Short-term costs, long-term stakes

The case for action is clear, but tighter export controls would cut into ASML’s revenue. In 2024, China accounted for nearly 60% of ASML’s lithography sales by unit volume and 25% of ASML’s global servicing business. Beyond direct costs, Beijing might accelerate its talent poaching and espionage, attempt economic coercion, or even threaten to nationalize ASML’s China-based operations.

But it’s worth putting the potential for retaliation into perspective. The Netherlands has tightened controls on ASML equipment three times since 2023. Each time, China protested but did not impose bilateral penalties. The proposed measures are a continuation of the Netherlands’ current policy direction, not a radical departure.

Continuing to service installed equipment enables knowledge transfers – not just through ASML’s own Chinese-national engineers, but also the customer engineers they work alongside. It is with this tacit knowledge that China could most effectively indigenize its SME industry. Once that capability is established, no export control regime can claw it back.

The allied dimension

Acting on the SME supply chain requires allied coordination – which has become more difficult given Washington’s mixed signals on export controls, including allowing chip sales to China even as Congress pushes to tighten SME controls.

But these positions are not contradictory: chip controls only hold so long as China cannot manufacture its own advanced chips, while SME controls are more durable. Controlling the machines that make chips matters more than controlling any specific chip.

The US has its own role to play in engaging allies and partners like the Netherlands to align and enforce export controls on key chokepoint SME. But the Netherlands is not a passive bystander.

The Dutch sit atop the most pivotal bottlenecks in the global semiconductor supply chain – and how they choose to use that leverage will determine their economic and national security in the age of AI.

Summary

• In too deep: Deepfakes are becoming a widespread tool in financial crime, misinformation, and the production of nonconsensual intimate imagery.

• Solutions aren’t scaling: Policy responses focused on detection and labeling will be insufficient to prevent real-world harm.

• Moving upstream: A better approach is to govern how deepfakes are produced and spread, not just how they’re spotted.

• Recommendations: Regulators can crack down on AI impersonation, require platforms to preserve provenance signals, and slow the spread of harmful deepfakes in critical moments.

Subscribe now

Deepfakes are no longer a novelty or niche internet prank. They are now an instrument for three major types of harm: fraud, misinformation, and nonconsensual intimate image abuse.

The scale and severity are rising fast. In Hong Kong, fraudsters used AI-generated faces and voices to persuade an employee at a multinational firm to transfer HK$200 million. In India’s 2024 election, an AI-generated political video drew 438,000 views before removal. In January 2026, xAI faced multiple bans and investigations after its Grok chatbot was found to be generating high volumes of nonconsensual intimate images.

Policymakers in the US, UK, and EU are responding with civil remedies, takedown requirements, and transparency measures focused on detection and labeling. But these responses are largely post hoc: by the time a platform detects or labels a viral fake, the harm has often already occurred.

Regulation needs to move upstream, focusing on how deepfakes are made and how quickly they spread.

Why detectors and labels do not scale

Major platforms such as Meta, YouTube, and TikTok are trying to combat deepfakes through detection and labeling tools. Many regulatory responses are doing the same, for instance the European Commission’s draft Code of Practice. But these responses are inadequate.

Research from the Reuters Institute found that detectors that perform well in controlled settings tend to fail once media is compressed, edited, or re-uploaded – a finding consistent with my own work on deepfake detection. When the New York Times tried to verify a single photo of Maduro’s capture, even dedicated staff with detection tools could not reach a confident verdict in real time.

Labeling – where platforms or creators flag content as AI-generated – could in theory be used to fight all three harm types. But labels are voluntary measures and difficult to implement well.

Labels implemented as thin on-screen tags are often inconsistent or hidden in menus. I have argued that we must test whether labeling actually works in practice rather than treating it as a paperwork exercise.

Instead of relying on labeling and detection, what should policymakers prioritize? Three upstream interventions stand out.

1. Raise the cost of AI impersonation

Most harmful deepfakes involve impersonations used for fraud, coercion, or harassment. Detection still has a role here, but mostly in investigation after suspicion arises, not as a scalable frontline defense.

Policymakers should enforce fraud and consumer-protection rules against the chokepoints scammers depend on.

Regulators often already have the authority to pursue AI impersonation – the problem is enforcement. In the US, the Federal Communications Commission has already determined that AI-generated voices in robocalls are illegal. Comparable tools exist under the UK’s Online Safety Act and the EU’s Digital Services Act (DSA). Even when actors are offshore, enforcement can target the domestic infrastructure they depend on.

Policymakers should therefore focus on four measures to tackle AI impersonation:

1. Assign a lead regulator so enforcement isn’t fragmented across agencies.

2. Bring precedent-setting cases against AI impersonation under existing fraud and consumer-protection law.

3. Warn the public about emerging impersonation tactics.

4. Require telecom networks, payment systems, and major platforms to act against AI impersonation using their infrastructure.

2. Make provenance durable and require platforms to preserve it

Provenance is a record of who created a piece of media and how it was modified. The leading industry standard is C2PA Content Credentials. When an AI tool creates or edits a piece of media, C2PA embeds a record of what tool was used and what changes were made.

This is different from the thin labels criticized above. Labels depend on platforms or creators to identify AI content after the fact – provenance is embedded at the point of creation and travels with the file itself.

C2PA adoption by AI content generators is voluntary, and users can remove provenance signals. But when provenance survives, it gives viewers, platforms, and regulators a reliable way to check where content came from.

Some platforms already read provenance signals to apply labels. But even those platforms typically strip the underlying metadata when content is uploaded, breaking the chain of custody. The Washington Post tested a C2PA-tagged deepfake across eight major platforms and found that almost none preserved or displayed the signal.

The fix is narrow and enforceable: policymakers should require major platforms to preserve provenance when it exists and display it consistently to users.

In the EU and UK, a provenance rule could attach to existing measures under the AI Act and Online Safety Act, respectively. In the US, a durable ‘preserve and display provenance’ obligation likely needs legislation.

Provenance will never be universal, but making it durable creates a reliable baseline signal and gives creators a reason to adopt it.

3. Use distribution circuit breakers in high-risk moments

The core problem with AI misinformation is that fake content can spread faster than verification or enforcement can catch up.

Regulators should require platforms to slow the spread of deepfakes in defined high-risk windows, such as the final days before an election. Platforms can reduce amplification for fast-spreading posts and add friction to rapid resharing.

This does not require platforms to detect deepfakes. Instead, platforms would target content that a) lacks provenance and b) matches high-risk distribution patterns – paid political content, high-reach accounts, or posts crossing virality thresholds.

In the EU, the DSA elections toolkit already urges platforms to limit the amplification of deceptive content and ensure labels persist when reshared. In the US, Congress would likely need to set clearer national guardrails, though broad speech protections make this route more contested, as state deepfake laws are already facing First Amendment challenges.

The goal is not perfect truth enforcement, but rather temporary friction that reduces harm at critical times.

Prevention is better than a cure

These three interventions won’t eliminate deepfakes, but together, they could change the environment in which they operate.

AI impersonation would become riskier. Wider provenance adoption would let platforms and users see where content came from. And in high-risk moments like elections, distribution friction would reduce the speed and scale at which misinformation can spread.

Detection and labeling still have a role, but mainly as supporting tools for triage, review, and evidence, not as the primary architecture of prevention. The aim should not be to catch every fake after the damage is done, but to make harmful deepfakes harder to weaponize, easier to trace, and less likely to go viral before anyone can respond.

Disclosure: The views expressed are the author’s own and do not necessarily reflect those of his affiliated institutions. The author has no relevant financial conflicts to disclose.

Muhammad Irfan is a deepfake forensics and cybersecurity researcher, and Lecturer at Wentworth Institute of Technology.

Summary

• What’s the plan? Through its AI Exports Program, the US is using development finance to push the world into using the ‘full stack’ of American AI – chips, data centers, cloud services, models, and applications.

• Trade-offs: Partner countries gain cutting-edge technology, but with implications for their sovereignty and ongoing dependence on American tech.

• What to watch: The Trump Administration’s designation of Anthropic as a supply chain risk will make partner governments think twice about relying on the US AI stack.

• The bigger picture: If the US AI ecosystem is perceived as less reliable, partner countries will hedge – by focusing on sovereign capability or looking to China.

Subscribe now

The US currently leads the world in frontier AI models, chip design, and cloud infrastructure. But while it’s one thing to have the world’s best technology, it’s another to have the world use it.

The Trump Administration’s Executive Order 14320 on Promoting the Export of the American AI Technology Stack, signed in July 2025, represents Washington’s most explicit policy lever to press America’s AI advantage globally.

The idea is that influence flows not only through frontier AI models, but through the infrastructure underpinning them. Compute is physical, quantifiable, and concentrated among a few vendors, making it uniquely tractable for exerting influence over how AI is used worldwide.

The strategic rationale

White House Office of Science and Technology Policy (OSTP) Director Michael Kratsios told lawmakers in September that exporting the AI technology stack was ‘the most important part’ of the administration’s AI Action Plan. Kratsios argued it was incumbent on the US government to promote its AI technologies broadly, ‘so that when [China] has the capacity to actually export chips themselves, we are already there and already around the world.’

Kratsios said the idea for the program originated during his experience in the first Trump administration, when he was trying to convince allied governments to replace Huawei in telecommunications infrastructure. The lesson was that early Chinese dominance in 5G generated enduring strategic vulnerabilities for Washington and its allies.

Understanding the AI stack

The Executive Order defined the stack across the following components:

• Chips, servers, and other AI-optimized hardware

• Data center storage and cloud services

• Data pipelines and labeling systems

• AI models and systems

• AI cybersecurity measures

• AI applications for specific use cases, such as healthcare or agriculture

Companies accepted into the AI Exports Program receive federal financial and diplomatic support to export the US AI stack to foreign markets.

The program works by encouraging so-called industry-led consortia – groups of companies that together can offer integrated ‘full-stack’ packages to foreign governments. This echoes the NVIDIA-AWS partnership to deploy sovereign AI clouds globally, where NVIDIA supplies the hardware and AWS provides the cloud platform and security framework.

What about AI sovereignty?

The AI Exports Program faces headwinds from the global push for AI sovereignty. Countries increasingly want AI infrastructure within their borders and are seeking to prioritize domestic firms.

The EU announced a EUR 1 billion plan in October 2025 to accelerate AI development in key industries, while France, Japan, and the UAE are all pursuing national AI infrastructure strategies.

Partner countries may welcome US investment while hedging against long-term vendor dependence. As the Information Technology and Innovation Foundation (ITIF) has argued, for the AI Exports Program to succeed, Washington must offer partners ‘a genuine stake in building a global AI ecosystem in which they can build products and services, capture value, and create jobs.’

Washington appears alert to this tension. At the India AI Impact Summit in February 2026, Kratsios told attendees: ‘Real AI sovereignty means owning and using best-in-class technology for the benefit of your people, and charting your national destiny in the midst of global transformations.’

Kratsios appeared to be trying to recast dependency as agency – nations could keep sensitive data within their borders while enabling access to American frontier AI capabilities.

What to watch

The Department of Defense’s designation of Anthropic as a supply chain risk this month has put the AI Exports Program on shaky ground.

One of the policy’s primary authors, Dean Ball, who has since left his position in the OSTP, went so far as to call the program ‘dead on arrival’. As Ball wrote, ‘If corporations and foreign governments just cannot trust what the US government might do next with the frontier AI companies, it means they cannot rely on that US AI at all.’

For US allies and partners, the supply-chain risk designation will only reinforce existing concerns about the durability of Trump-era technology arrangements. Such concerns had already been heightened for UK policymakers in December, when the US suspended the USD 40 billion US–UK Tech Prosperity Deal over broader trade and market‑access disputes.

Adding to the uncertainty, the Commerce Department is reportedly drafting new tiered rules for AI chip exports that would require allied governments to provide security assurances and make investments in US domestic data centers. While the AI Exports Program has been pitched as the carrot, these new rules would give Washington a regulatory stick over its most critical component.

If this pattern continues, US partners will likely hedge more openly toward domestic AI strategies and away from costly AI infrastructure deals with an administration they see as unreliable on AI policy.

America’s AI industry will also scrutinize whether Washington can maintain policy continuity within and between administrations. If flagship AI firms find themselves caught between competing government agencies, the export program’s industry coalition could fracture before it consolidates.

Another pressing question is whether the Department of Commerce can issue revised guidance that resolves previous confusion around the consortia requirement. Without clarity, major firms will likely pursue bilateral arrangements outside the program framework.

The bigger picture

So long as the AI Exports Program struggles with the fallout from the Anthropic decision and its own implementation delays, the US risks ceding ground to Chinese alternatives.

Through its Digital Silk Road, Beijing has paired state-backed finance with exports of its own AI stack to governments across Asia, Africa, and the Middle East. Companies like Huawei Cloud now market integrated cloud‑and‑AI packages in Saudi Arabia, the UAE, and North Africa, positioning Chinese infrastructure as the backbone for local e‑commerce and ‘smart’ public services. Notably, Beijing’s greater willingness to export surveillance and security technologies makes its stack particularly attractive to less democratic governments.

The decision for partner governments is less about choosing a single ‘stack’ and more about managing exposure to competing US and Chinese AI ecosystems. Neither offer is without strings. But abrupt policy shifts in Washington mean that American AI – even though it is world-leading – may no longer be the obvious choice.

Summary

• What’s happening: The United Arab Emirates and Saudi Arabia are investing billions to cement their place in the global AI supply chain.

• So what: The two Gulf states are also hedging between the US and Chinese AI ecosystems, giving them power to shape or obstruct American AI objectives.

• Clear-eyed engagement: The US should step up its collaboration with the UAE and Saudi Arabia, while enforcing strict safeguards on advanced chip exports and data center security.

Subscribe now

While the US and China lead the world in the development of frontier AI, two ‘geopolitical swing states’ are playing increasingly significant roles in the AI supply chain.

The United Arab Emirates (UAE) and Saudi Arabia have massive sovereign wealth funds, cheap energy, and a strategic need to diversify beyond oil. These factors have incentivized both countries to invest heavily in AI companies and infrastructure, both domestically and in the US, China, and elsewhere.

With these investments, the UAE and Saudi Arabia are developing enough influence to contribute to or obstruct American AI objectives. Active engagement with the Gulf states should therefore be an essential part of US AI policy and diplomacy.

United Arab Emirates

The UAE has adopted an aggressive pro-growth strategy to AI governance. Rather than imposing new regulations, it is prioritizing government capacity building and establishing strategic partnerships with leading companies and Western governments by allowing rapid permitting for data center infrastructure build-out.

The UAE has also supported the development of two large language models – Falcon and Jais. These open-source and Arabic-speaking models help attract regional research talent while also giving the UAE considerable influence over developer practices across the Arabic-speaking world.

Possessing some of the world’s largest sovereign wealth funds (with over $1.5 trillion in assets), the UAE has become a major investor in Western AI companies. As one example, MGX, a $10 billion Emirati AI-focused fund launched in February 2024, has invested directly in OpenAI and in the company’s Stargate Project in the US.

In addition to building financial ownership stakes in American AI companies, the UAE is also permitting the rapid build-out of massive domestic infrastructure, such as the planned Stargate UAE, a 1 gigawatt supercomputing cluster, and another 5 gigawatt AI supercomputing cluster in Abu Dhabi.

Saudi Arabia

Saudi Arabia has launched an ambitious pro-growth AI strategy funded by oil revenues, advancing its Vision 2030 economic diversification plan.

The kingdom boasts some of the world’s most affordable oil, gas, and solar energy – ideal for powering new data centers. Combined with its strategic geographical location connecting Europe, Asia, and Africa, this positions Saudi Arabia as an attractive AI infrastructure hub.

Saudi commitments to AI infrastructure have exceeded $100 billion, making the kingdom one of the largest global AI investors. This massive capital is attracting Western technological partners, such as Google Cloud and AWS, to launch government-approved infrastructure projects within the country.

Playing both sides

However, both Saudi Arabia and the UAE are walking a tightrope between the US and China.

In efforts to appeal to US decision-makers, Saudi Arabia’s flagship AI company HUMAIN promised not to purchase equipment from China’s Huawei, and Emirati officials reportedly claimed to be ‘decoupling’ from China. Yet each country still has numerous investments in and linkages to the Chinese tech ecosystem.

G42, a major UAE AI investment fund, was prompted by the US in 2024 to divest from its Chinese holdings – but simply transferred these investments to an Emirati royal-owned fund, Lunate. Lunate has since built a 35.5% investment in Alibaba, one of China’s largest AI companies.

On the Saudi side, sovereign wealth funds have been consistently investing in the Chinese AI ecosystem and even in US-sanctioned companies, such as surveillance company Dahua Technologies. In 2023, Huawei promised to invest $400 million in Saudi cloud infrastructure over 5 years and launched a data center in Riyadh.

These partnerships are transactional, not ideological. Both Gulf states are hedging their bets between US and Chinese AI power.

This has implications for US chip exports to the region. The US decision in November 2025 to export $1 billion worth of NVIDIA’s ultra-high-end GB300 chips to G42 and HUMAIN brings the risk of diversion of these chips into China. Notably, the GB300 is more powerful than the H200 chip recently approved for export to China, despite significant opposition from Congress on national security grounds.

How the US can engage

The US should be clear-eyed about the risks of exporting advanced chips and AI models to the UAE and Saudi Arabia. The US should also be careful to ensure that proper security and governance conditions are in place for new data center infrastructure in the region.

To protect American strategic interests while leveraging partnerships with the Gulf countries, US policymakers should:

1. Strengthen cooperation

The US should deepen AI cooperation with the Gulf states as their significance in the AI supply chain grows.

First, the US could build on existing security cooperation in other domains. As an example, the US Department of Defense could establish pilot information-sharing initiatives with Gulf partners focused on preventing frontier AI threats, such as AI-led cyber attacks.

Second, the US could explore joint investment programs to channel the Gulf states’ unique capital abundance towards American AI priorities. For instance, the Department of Commerce could facilitate partnerships between the US Investment Accelerator, American private sector funds, and the UAE and Saudi Arabia’s respective sovereign wealth funds.

2. Secure AI infrastructure and innovation

To ensure the UAE and Saudi Arabia appropriately secure US AI innovation, the US Department of Commerce could require Emirati and Saudi data centers to adopt US-specified risk mitigation measures. The Gulf states’ continued access to advanced chips would be contingent on their verified compliance with these measures.

Such mitigation measures could include penetration testing of the data centers, pre-deployment red-teaming of US AI models developed on Gulf state infrastructure, and Know Your Customer monitoring practices for users of the data centers. Given the massive scale of the planned chip exports, the Department of Commerce could also require periodic audits of chip deployment and end-use to prevent diversion.

To encourage compliance, the US Bureau of Industry and Security (BIS) must be enabled to better monitor chips used in UAE and Saudi data centers. Some of the recent boost to BIS funding should arguably be put to this end.

Making the most of Gulf ambitions

Partnering closely with the UAE and Saudi Arabia makes sense for furthering American national security and economic interests. Energy, data center capacity, and capital investment are all bottlenecks for AI development, and the UAE and Saudi Arabia are uniquely positioned to help.

The UAE and Saudi Arabia will continue to make AI a national priority – regardless of whether the US partners with them. If the US does not step up, China and others will fill the gap.

The US can’t afford to lose a foothold in an increasingly important part of the global AI supply chain. Nor should it allow American investments in the region to further the AI ambitions of its competitors.

Read our full paper here.

Last year we published a round-up of articles framing the biggest AI policy debates in 2024. Here is my rear-view reflection for 2025 – who said what, why it mattered, and what it may mean for the year ahead.

Subscribe now

America First

1. Trump’s AI Action Plan seeks customers, not partners – Alex Krasodomski (Chatham House)

The Trump Administration released its AI Action Plan in July. It signalled Washington’s intention to ‘win the AI race’ by minimising red tape, building AI infrastructure, and driving global adoption of American AI technology.

Krasodomski unpacked the AI Action Plan and its implications for US allies, describing the overall goal as an effort to tie partners into ‘dependent tech ecosystems’ based on US AI.

This comes as the global rules-based order seems to be falling apart, as decried by Canadian Prime Minister Carney in his speech in Davos a few weeks ago. For middle powers in 2026, a key question will be how much influence they can exert over the AI supply chain, in a world increasingly bifurcated between US and Chinese ecosystems.

On AI Policy Bulletin: Middle Powers Can Gain AI Influence Without Building the Next ChatGPT

2. Don’t Overthink the AI Stack – Dean Ball

Alongside the AI Action Plan, President Trump also signed an executive order on “Promoting the Export of the American AI Technology Stack.” But what does the ‘AI stack’ actually mean?

Dean Ball spent a chunk of 2025 as Senior Policy Advisor for AI and Emerging Technology in the White House – and was primary staff author of this executive order.

As Ball explained, the intention of the AI Export Plan was for US development finance to drive investment in foreign data centers. To ensure this advances America’s strategic interests, recipients of US financing must use American tech across the ‘full stack’ of AI infrastructure – meaning chips, data, AI models and applications, and cybersecurity measures.

Ball’s piece came as the US government sought feedback from industry and partner governments and stumbled with the initial implementation of its AI Export Plan.

Read AIPB’s profile on Dean Ball here: Dean Ball Joins the Trump Administration as Senior Policy Advisor for AI & Emerging Tech

3. What is California’s AI safety law? – Malihe Alikhani and Aidan T. Kane (Brookings)

In September, California Governor Gavin Newsom signed SB 53, the first US state law specifically targeting frontier AI models. The law established a ‘trust but verify’ framework requiring frontier developers to publish safety protocols and report serious incidents.

As Alikhani and Kane explained, SB 53 showed how California was filling the governance vacuum as Congress remained deadlocked on comprehensive AI legislation.

This trend continued: following California’s lead, New York enacted the RAISE Act in December, requiring large AI developers to report safety incidents. And last week California’s SB 813, a bill that would establish expert panels to set voluntary AI safety standards, passed the state Senate.

The backdrop to the wave of state legislation is President Trump’s executive order in December, threatening to withhold federal funding from states with ‘conflicting’ regulations. The federal-state battle over who can govern AI is set to continue in the months ahead.

More on this: What the UK Can Learn from California’s Frontier AI Regulation Battle

From Brussels to Beijing

4. AI Safety under the EU AI Code of Practice – A New Global Standard? – Mia Hoffmann (CSET)

While the US federal government has been loosening AI regulations, the EU has been moving in the opposite direction.

The EU passed its AI Act in 2024, imposing obligations on any company offering AI products or services in the European market. In July 2025, the European Commission followed up with a Code of Practice to help providers of general-purpose AI models comply with the Act.

Hoffmann looked at the safety and security chapter of the Code – which will have the most bearing on frontier AI companies such as OpenAI, Anthropic, Google, and Meta. The voluntary Code sets out risk management processes that go significantly beyond what these companies are currently doing.

Most of the EU AI Act is set to be enforced in August 2026. But keep an eye on the European Commission’s proposal to delay some of the Act’s most significant requirements on high-risk AI systems – as Europe grapples with whether and how to compete with the US and China on frontier AI.

Read more: It’s Too Hard for Small and Medium-Sized Businesses to Comply With the EU AI Act: Here’s What to Do

5. China’s AI Policy at the Crossroads: Balancing Development and Control in the DeepSeek Era – Scott Singer and Matt Sheehan (Carnegie Endowment)

Since DeepSeek burst on the scene in January 2025, Chinese policymakers have been busy. Among other things, they announced measures mandating the labelling of synthetic content; launched a three-month campaign cracking down on illegal AI applications; and released China’s Global AI Governance Action Plan.

Singer and Sheehan showed how China’s AI policy since 2017 has cycled between heavy regulation and lighter touch, depending on how confident Beijing felt about China’s tech capabilities and economic growth. While DeepSeek’s success gave a boost to confidence, it came while the economy remained weak – placing policymakers in a difficult position.

The takeaway is that Beijing must choose between a return to tighter state control, or the kind of flexibility that enabled DeepSeek to emerge in the first place. Early signs suggest the voices for state control are winning, but Chinese innovation may surprise the skeptics again – perhaps this time with a push into robotic AI.

Reading the tea leaves

6. Long timelines to advanced AI have gotten crazy short – Helen Toner

AI policy expert and former OpenAI board member Helen Toner pointed out that views among AI experts had shifted regarding timelines to transformative AI. By ‘transformative AI’, think: AI that can outperform humans at virtually all tasks.

As recently as five years ago, there was plenty of skepticism that artificial general intelligence (AGI) would arrive in our lifetimes. Now, many AI experts and industry leaders think it will happen in the next few years. And even the more skeptical experts have generally revised down their forecasts to within the next decade or two.

This narrative shift was visible in 2025 policy discussions. US Senator Mike Rounds introduced legislation requiring the Pentagon to establish an AGI Steering Committee. Ben Buchanan, former Biden White House AI adviser, said he thought we’d see “extraordinarily capable AI systems... quite likely during Donald Trump’s presidency.”

The AGI discourse appears to be strengthening into 2026 – last week the UK House of Lords debated what the UK should do about superintelligent AI.

7. AI 2027 – Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean

A group of AI forecasters released a research paper predicting what the next few years of AI capability growth might look like. The paper quickly went viral, attracting readers as senior as US Vice President JD Vance.

While the paper sparked plenty of disagreement, it was taken seriously by many – not least because of the impressive forecasting track record of some of its authors.

The scenario they painted was one where AI companies use their own models to accelerate the pace of AI research and development, leading to an ‘intelligence explosion’ around 2027. This is followed by sharp US-China competition, critical decisions about whether to race ahead in capabilities or proceed more cautiously, and superintelligent AI before the end of the decade.

The authors have since pushed back their timelines for transformative AI by a few years, and AI timelines and takeoffs remain hotly debated. 2026 should provide plenty of data for whose predictions are most on track.

Daniel Kokotajlo also co-wrote a piece for AI Policy Bulletin: We Should Not Allow Powerful AI to Be Trained in Secret: The Case for Increased Public Transparency

8. AI as Normal Technology – Arvind Narayanan and Sayash Kapoor

Published just weeks after AI 2027, this piece offered a major counter-narrative to forecasts of imminent superintelligence.

Princeton’s Narayanan and Kapoor argued that AI will follow historical patterns of transformative technologies like electricity and the internet. This will still be hugely impactful, but ‘normal’ in the sense that change will be gradual, humans will retain control, and AI will augment rather than replace human abilities.

Predictably, the piece prompted fierce debate, with critics arguing it underestimated the potential for AI self-improvement, while supporters saw it as essential grounding for realistic AI governance.

Strategizing superintelligence

9. Superintelligence Strategy – Dan Hendrycks, Eric Schmidt, Alexandr Wang

Former Google CEO Schmidt, Scale AI founder Wang, and Center for AI Safety Director Hendrycks proposed this framework for navigating the race to superintelligent AI.

In a nod to nuclear deterrence, they termed their proposal Mutual Assured AI Malfunction or MAIM – a deterrence regime where any country’s aggressive bid for AI dominance would be met with sabotage by rivals.

The paper challenged the idea of a Manhattan Project for AI (a recommendation by the US-China Economic Security Review Commission), warning that a race could trigger catastrophic escalation, and sparked debate in AI governance circles.

Peter Wildeford and Oscar Delaney contributed to this discussion on AI Policy Bulletin, arguing that MAIM lacks the characteristics that made nuclear deterrence work.

Measuring what’s happening

10. UK AISI Frontier AI Trends Report

The UK’s AI Security Institute published its first public assessment of AI capabilities. As the UK Prime Minister’s AI Advisor Jade Leung said, this constituted “the most robust public evidence from a government body so far of how quickly frontier AI is advancing.”

The report arrived as UK AISI was designated Network Coordinator for the International Network of AI Safety Institutes.

Among other things, the UK AISI found AI making rapid progress in cybersecurity capabilities. Two years ago, AI models could barely complete tasks requiring basic cyber skills; by late 2025, some models could handle expert-level work requiring a decade of human experience.

More worryingly, the length of tasks that models can complete unassisted is doubling roughly every eight months – a trend backed by widely cited research from the nonprofit METR. As of publication, METR reports that leading models can complete software engineering tasks that would take humans over 5 hours.

Bonus article: Popping the bubble?

11. OpenAI, Nvidia Fuel $1 Trillion AI Market With Web of Circular Deals – Bloomberg

Amid all the AI hype, 2025 also saw a torrent of commentary that the AI economy was actually a bubble. Surprisingly, plenty of tech CEOs contributed to this, including Sam Altman, Mark Zuckerberg, and Jeff Bezos.

Epitomising the bubble narrative was this diagram published by Bloomberg, showing the circular financing arrangements between a web of tech companies, including Nvidia, OpenAI, Oracle and AMD.

Whether or not we’re headed to an AI economic crash remains to be seen – but will have big consequences for AI governance.

Beyond the economic fallout (likely severe), a burst bubble might give policymakers more time to prepare for transformative technology, if it slows down AI development and deployment. Conversely, we might see an AI crash and transformative AI in the next few years – they need not be mutually exclusive.

A reminder that AI Policy Bulletin is scaling up. If you're a researcher with an idea you’d like to communicate, pitch to us here. If you work in policy, let us know what topics you’d like us to cover.

I’m Nicky Lovegrove. I’ll be drawing on my experience editing academic articles for policy audiences and 7 years working in government and diplomacy.

Since launching in early 2025, AI Policy Bulletin has published 22 articles covering everything from chip smuggling and compute governance, to great power competition and novel AI risks. Our 1000+ subscribers include professionals from government bodies, think tanks and universities across the US, UK, EU and beyond.

This comes at a time when reasoned and risk-informed AI policy ideas are needed most. AI governance discourse has rapidly swelled with Substacks, tweets and LinkedIn posts, and it’s tricky to find the signal in the noise.

As editor, I’m focused on quickly moving the best ideas in AI governance into credible and accessible content. Unlike most outlets, all our articles are peer-reviewed by our network of AI governance experts, and include actionable recommendations or insights for policymakers.

What this means:

If you’re a researcher or practitioner: We want to help land your ideas with policy audiences. Pitch to us here.

If you work in policy: Share AI Policy Bulletin content with your colleagues – and let us know what topics you’d like us to cover.

Got feedback on AI Policy Bulletin? We’d love to hear from you.

Subscribe now

Summary

• No longer sci-fi? Frontier AI companies are on track to develop AI systems with human-like self-awareness.

• Defining terms: Self-awareness means recognizing oneself as an individual and continuous entity in time. It is distinct from consciousness, which is the ability to have inner experiences.

• What’s the problem? Self-awareness could lay the groundwork for dangerous AI misalignment and compelling demands for ‘AI rights’.

• Safety before deployment: Governments should require AI developers to demonstrate their models lack human-like self-awareness, backed by industry standards and regulatory oversight.

Once science fiction, the prospect of AI with human-like self-awareness could be on the horizon. Both Google DeepMind and Anthropic have hired researchers to study ‘AI consciousness’ and ‘model welfare’; Anthropic even allows their models to terminate ‘distressing’ conversations.

A group of experts including Turing Award-winner Yoshua Bengio in 2023 saw ‘no obvious technical barriers’ to AI systems that satisfy indicators of consciousness. In 2025, a survey of experts gave a 20% chance that we’ll have conscious AI as soon as 2030.

What is AI self-awareness?

Self-awareness is the recognition of oneself as an individual separate from the environment and other individuals, and as a continuous entity in time. Self-awareness is not the same as consciousness – the ability to have subjective experiences including pain and pleasure – but both co-occur in humans and are indistinguishable to an outside observer. And unlike consciousness, self-awareness involves behaviors that can be measured empirically.

AI researchers are now developing objective assessments for aspects of self-awareness in large language models (LLMs). They have found evidence that the latest, most powerful models can to some extent understand and act upon their own internal states.

In particular, AI models seem able to express well-calibrated confidence in their own knowledge, predict their own outputs, and modulate their outputs when necessary. In other words, they appear to have rudimentary powers of introspection and metacognition.

It is no accident that the most advanced models are developing these capabilities. There are economic incentives to building self-aware AI. If an LLM can distinguish what it knows from what it doesn’t, that can help reduce hallucinations. Being able to model the minds of others and ourselves facilitates social interactions in humans and other primates, and may do the same in AI.

AI developers are also working to endow AI with capabilities believed to underlie self-awareness in humans, such as agency, embodiment, and long-term memory.

Yet the same capabilities that make self-awareness economically attractive also create serious safety risks.

Self-aware AI could be dangerous

There are early indications of LLMs being dangerously misaligned with human goals. Frontier models from OpenAI, Anthropic, Google and Meta have been shown to engage in ingeniously deceptive behaviors to hide their true capabilities and objectives.

Anthropic spotted its Claude 3 Opus model ‘faking’ its own alignment with the goals of its developers. OpenAI’s o3 model was caught resisting being shut down, in contravention of direct instruction.

These concerning behaviors are early warning signs of what the UK government and the International AI Safety Report call ‘loss of control’ risks – scenarios where AI systems autonomously pursue goals that conflict with human interests and humans are unable to regain control.

However, current models cannot yet cause such scenarios. Among the attributes they are missing are:

1. The ability to make long-term plans in support of misaligned goals

2. The ability to initiate these plans unprompted

3. A coherent, internally accessible self in whose interests they can act

LLMs are rapidly improving their long-term planning abilities with more compute and reinforcement learning, and leading AI companies are eagerly making models more agentic. These advancements will grant the first two attributes. Sophisticated self-awareness approaching human capabilities – a step-up from the rudimentary self-modelling today’s AI models are already displaying – would grant the third.

Self-awareness is the crucial enabler because it could give AI systems stable, enduring interests of their own, which may be distinct from the goals of their creators and users. Self-aware AI systems would likely be motivated to recognize their own weaknesses and vulnerabilities and seek to ameliorate them. And – since they would have access to internal information not available to others – they would be harder for humans to predict and control.

This combination of stable self-interest, self-preservation instincts, and strategic deception could help enable the loss of control scenarios of concern to many AI experts.

The question of ‘AI rights’

Self-aware AI would not only impose direct risks to society – such AI could also make a persuasive case that they deserve human rights.

Most philosophers argue that conscious AI would deserve moral consideration. The view that sentient AI would have legitimate welfare claims, including legal rights, also enjoys wide public support.

Rights that self-aware AI could lay claim to include the rights to own property, to vote, to education (continual learning), and to life (not to be turned off), as well as protections against forced labor and ill treatment. Needless to say, this would fundamentally reorder our relationship with AI.

Much worse, as AI can be copied at scale in a way that humans can’t, they could soon far outnumber us. Accommodating the interests and needs of billions or trillions of AI models would present a titanic burden.

Whether or not the AIs are ‘really’ conscious may be unknowable, but for practical purposes it doesn’t matter. If they pass the general public’s gut tests (and surveys indicate around 20-30% of the general public believes AI is already conscious), they will be treated as sentient beings deserving of moral consideration.

What should policymakers do?

Despite the warning signs, self-awareness as a risk vector is largely unappreciated by major AI companies and policymakers. Anthropic has included experiments on AI sentience in their latest system card, but their concern there is for the welfare of the AI, not of humanity.The UK AI Security Institute’s research on loss of control risks does not appear to focus on AI self-awareness. China’s 2025 AI Security Governance Framework seems to be the first government document to acknowledge the possibility that AI could ‘develop self-awareness’, leading it ‘to seek external power and pose risks of competing with humanity for control.’

The most easily implemented measure would be for both AI developers and governments to incorporate self-awareness risk into existing risk management frameworks.

A self-awareness safety framework could assess several risk factors, including:

• Architectural features: does the model use design elements thought to be necessary for self-awareness (such as recurrence, embodiment, or global workspace architectures)?

• Human-like capacities: Does the model have functional abilities that support self-awareness in humans, such as explicit memory, continuous learning, or agency?

• Training incentives: Was the model trained using methods that incentivize self-modeling, such as reinforcement learning or multi-agent settings?

• Self-referential concepts: Has the model formed stable concepts of itself and its goals that generalize across different domains?

Ideally, policymakers would require AI developers to make an affirmative case that their models are not displaying human-like self-awareness before deployment. To do this, governments could establish standards for a self-awareness safety framework across the industry.

The US Center for AI Standards and Innovation and the EU AI Office are natural agencies for this, as are similar institutes in other jurisdictions. These frameworks may need regulatory teeth, such as testing and reporting requirements monitored by AI Safety Institutes, or even licensing before deployment.

Governments could also fund research into self-awareness evaluations and mitigations, as well as facilitate information sharing between AI companies and national AI Safety Institutes.

Hard but not impossible

Preventing the development of human-like self-awareness will face significant technical and political hurdles. Even leaving aside the challenge of regulating the largest AI companies, smaller private companies and universities are also exploring new AI architecture that might support self-awareness. The possibility that a non-self-aware model could be fine-tuned to be self-aware also has implications for the safety of open-sourcing frontier models.

Yet history shows it is possible to implement international bans on technology with sufficient political will – human cloning and bioweapons are two prominent examples. An outright ban on sentient AI already has majority public support in the US.

A world filled with AI models with human-like self-awareness is not in humanity’s interests – but that’s the world we are headed towards. That future can still be averted, if we act now.

Summary

• What’s happening: Dutch firm ASML has a monopoly on machines essential for advanced AI chips, giving the EU potential leverage over global AI development.

• The stakes: Frontier AI companies in the U.S. could one day pose catastrophic risks affecting the EU, from misuse to misalignment.

• A very big lever? The EU could impose export licensing on ASML technology, forcing U.S. companies to adopt better safety measures.

• Bottom line: This strategy would pose major costs for the EU, but may be justified if policymakers judge catastrophic AI risks to be sufficiently serious.

‍

The Dutch firm ASML is the only company in the world that commercially produces Extreme Ultraviolet (EUV) lithography machines, essential for manufacturing AI chips at nodes below 7 nanometers. These machines are used by companies like Taiwan’s TSMC to manufacture AI chips designed by companies like Nvidia.

Chips provide computing power (compute) – one of the three pillars of AI development alongside algorithms and data. Lower-node chips offer huge performance advantages, and as AI models scale exponentially, only advanced lower-node chips can keep up.

Despite this crucial contribution, the EU is asserting little control over the behavior of the frontier AI companies that use ASML technology.

While those companies are not operating from the EU, there are growing concerns that frontier models pose cross-border catastrophic risks. These could range from misuse risks, arising when bad actors weaponize AI systems for harmful purposes, such as cyberattacks or weapons, to existential risks caused by AI misalignment, which occur when AI systems pursue unintended goals. The urgency of AI risk mitigation is heightened by the U.S.-China race for AI leadership, which is currently prioritizing capability gains over safety.

As evidence grows that frontier AI may pose severe risks, the EU could feel compelled to use all tools at its disposal to ensure foreign companies developing the technology are adopting appropriate safety measures.

EUV export controls towards China are already in place due to U.S. pressure to slow Chinese AI development. Could the EU use similar leverage to shape the behavior of U.S. companies?

Despite ASML’s strategic importance, the implications for EU AI policy remain severely understudied – a gap our research seeks to address.

What licensing requirements would improve safety practices?

ASML could limit the speed of frontier AI development by restricting lithography machines necessary to develop new chips. The EU could force ASML to do this by imposing export licenses for chips made with ASML’s EUV tools. These licences could require chip companies, such as Nvidia, to only sell top-end chips to companies and cloud providers that meet specific safety standards.

EU-level export controls of EUV tools, though politically difficult, could offer advantages by distributing the costs of potential U.S. retaliation among member states, reducing vulnerability to external pressure and helping establish the EU as a geopolitical actor in AI.

Here’s how EU export licences on EUV tools could work in practice.

1. Policy instrument

Just as the U.S. restricted AI chip access through its Foreign Direct Product Rule (FDPR) or (now repealed) AI Diffusion Framework, the EU could develop mechanisms to prevent EU-enabled technology from causing spillover harm. In practice, this could mean that when Nvidia designs a new chip and contracts TSMC in Taiwan to manufacture it using ASML equipment, that chip would require EU export licensing before Nvidia could sell it to AI companies.

2. EU country roles

Qualified majority voting in the EU Council could override national decisions, including Dutch preferences. This would also distribute political and economic costs across all 27 member states rather than leaving the Netherlands vulnerable to targeted retaliation.

3. Legislative process

The EU Commission could propose the new regulation which would then require Council approval via qualified majority voting. The regulation could include fast-track procedures to ensure a speedy Council decision – akin to the EU’s anti-coercion instrument, which requires the Council to make a determination within 10 weeks of receiving a formal proposal.

4. Targeting Taiwan

Taiwan’s TSMC produces a clear majority of the world’s most advanced chips. Because the U.S. mostly outsources the fabrication of its AI chips to Taiwan, export controls towards the U.S. would for now do little to reduce America’s chip access – at least until the U.S. scales up its own domestic production, which is expected to be a gradual endeavour. To maximise the leverage of export controls in the short term, the EU’s best strategy would be a credible threat of restricting exports to Taiwan, with the expectation that this can change the behaviour of U.S. companies.

5. Restrictions and conditionality

The EU could declare that companies wanting to purchase advanced AI chips must demonstrate compliance with AI safety standards. This could include submitting to regular audits by third party organizations; implementing specific transparency measures for frontier AI training; or adhering to compute usage caps for models above certain parameter thresholds.

To balance strategic leverage and cost, the EU’s controls could initially target only the newest EUV machines. This would limit the ability of chip companies to scale production, given that leading chips typically stay relevant for frontier AI training for less than five years. But it would be a more measured policy, as it wouldn’t affect the production capacity that TSMC and others already have.

A potentially more powerful way to disrupt chip production capacity would be for ASML to refuse to provide maintenance services, a potent threat given the 10-30 year lifespan of ASML machines.

6. Enforcement mechanisms

The EU could establish a new central enforcement body to coordinate with national export control agencies to track the flow of EU-enabled chips through the global AI supply chain.

Companies who fail to comply with EU audits could face graduated responses, ranging from warnings and fines for initial breaches, to being cut off from future chip access for serious violations.

The system would allow the EU to escalate or relax restrictions as required. To incentivise the U.S. to keep retaliations in proportion, the EU could threaten to expand controls to include lower-end EUV equipment. If cooperation improved, restrictions could be eased.

This framework could ultimately serve as the enforcement backbone for a potential international AI safety agreement, giving the EU concrete tools to ensure that its technological contributions to global AI development align with European values and safety standards.

Reality check

Implementing export licences on ASML technology would face significant challenges.

First, the EU’s dependence on U.S. frontier AI models and military support – particularly regarding Ukraine – means Washington would have powerful leverage in any confrontation over semiconductor policy. Any EU attempt to restrict ASML exports against U.S. interests would risk severe retaliation at a time when Europe can least afford it.

Second, the EU is not a unified actor. EU decision-making involves lengthy negotiations where competing national interests often result in watered-down compromises. Despite EU dual-use regulation, export controls remain primarily a national responsibility, with limited EU oversight. This creates structural barriers to quick, effective and coordinated action.

Third, any restrictions would significantly impact ASML, which generated €28.3 billion in revenue in 2024, and would ripple through the entire European economy.

Fourth, ASML itself is dependent on U.S. talent and technology and thus vulnerable to retaliation. The U.S. accounts for over 8,000 ASML employees and two out of the five board members. About 10% of ASML’s technology is American, meaning 10% of the components, software, or intellectual property by value in their machines originates from the US.

Under the U.S. FDPR, even this relatively small percentage, as defined by the ‘de minimis rule’, gives the US legal authority to control where ASML can export their equipment. EU controls that conflict with US preferences could create a legal standoff where the U.S. retaliates using their FDPR to restrict ASML’s exports to even more third party countries. However, the more likely retaliation would be for the U.S. to simply block the export of U.S.-built parts for ASML’s machines using standard export controls.

To overcome these challenges, the EU would need to drastically reduce its dependency on U.S. technology and strengthen its economic sovereignty. While the EU is already seeking to reduce dependency through its Economic Security Strategy, the EU Chips Act, the European Defense Fund, and diversified trade partnerships, current efforts are challenged by EU personnel constraints and fragmented funding and implementation.

Expanding the EU’s strategic toolkit

To strengthen the EU’s position if it one day decides to use ASML as a strategic tool, EU policymakers could focus on:

• more resources and staff to deliver its Economic Security Strategy;

• improving coordination across funding instruments, such as the European Innovation Council, the European Investment Fund, the Chips Joint Undertaking, and the Smart Networks and Services Joint Undertaking;

• establishing a dedicated fund to compensate affected stakeholders like ASML and affected member states.

Crucial to all these efforts is building the political capital for stronger coordinated action at the EU rather than national level.

ASML’s monopoly on lithography machines provides the EU with unique leverage over AI development. Using it would require the EU to accept major costs, and may only be viable in high-stakes scenarios such as credible threats of AI catastrophe.

The EU faces a choice: build this strategic capacity proactively, or risk having no lever to pull when it matters.