Deepfake Policy Is Focused on the Wrong End of the Problem

Efforts to detect and label deepfakes are fighting a losing battle.

Summary

• In too deep: Deepfakes are becoming a widespread tool in financial crime, misinformation, and the production of nonconsensual intimate imagery.

• Solutions aren’t scaling: Policy responses focused on detection and labeling will be insufficient to prevent real-world harm.

• Moving upstream: A better approach is to govern how deepfakes are produced and spread, not just how they’re spotted.

• Recommendations: Regulators can crack down on AI impersonation, require platforms to preserve provenance signals, and slow the spread of harmful deepfakes in critical moments.

Deepfakes are no longer a novelty or niche internet prank. They are now an instrument for three major types of harm: fraud, misinformation, and nonconsensual intimate image abuse.

The scale and severity are rising fast. In Hong Kong, fraudsters used AI-generated faces and voices to persuade an employee at a multinational firm to transfer HK$200 million. In India’s 2024 election, an AI-generated political video drew 438,000 views before removal. In January 2026, xAI faced multiple bans and investigations after its Grok chatbot was found to be generating high volumes of nonconsensual intimate images.

Policymakers in the US, UK, and EU are responding with civil remedies, takedown requirements, and transparency measures focused on detection and labeling. But these responses are largely post hoc: by the time a platform detects or labels a viral fake, the harm has often already occurred.

Regulation needs to move upstream, focusing on how deepfakes are made and how quickly they spread.

Why detectors and labels do not scale

Major platforms such as Meta, YouTube, and TikTok are trying to combat deepfakes through detection and labeling tools. Many regulatory responses are doing the same, for instance the European Commission’s draft Code of Practice. But these responses are inadequate.

Research from the Reuters Institute found that detectors that perform well in controlled settings tend to fail once media is compressed, edited, or re-uploaded – a finding consistent with my own work on deepfake detection. When the New York Times tried to verify a single photo of Maduro’s capture, even dedicated staff with detection tools could not reach a confident verdict in real time.

Labeling – where platforms or creators flag content as AI-generated – could in theory be used to fight all three harm types. But labels are voluntary measures and difficult to implement well.

Labels implemented as thin on-screen tags are often inconsistent or hidden in menus. I have argued that we must test whether labeling actually works in practice rather than treating it as a paperwork exercise.

Instead of relying on labeling and detection, what should policymakers prioritize? Three upstream interventions stand out.

1. Raise the cost of AI impersonation

Most harmful deepfakes involve impersonations used for fraud, coercion, or harassment. Detection still has a role here, but mostly in investigation after suspicion arises, not as a scalable frontline defense.

Policymakers should enforce fraud and consumer-protection rules against the chokepoints scammers depend on.

Regulators often already have the authority to pursue AI impersonation – the problem is enforcement. In the US, the Federal Communications Commission has already determined that AI-generated voices in robocalls are illegal. Comparable tools exist under the UK’s Online Safety Act and the EU’s Digital Services Act (DSA). Even when actors are offshore, enforcement can target the domestic infrastructure they depend on.

Policymakers should therefore focus on four measures to tackle AI impersonation:

1. Assign a lead regulator so enforcement isn’t fragmented across agencies.

2. Bring precedent-setting cases against AI impersonation under existing fraud and consumer-protection law.

3. Warn the public about emerging impersonation tactics.

4. Require telecom networks, payment systems, and major platforms to act against AI impersonation using their infrastructure.

2. Make provenance durable and require platforms to preserve it

Provenance is a record of who created a piece of media and how it was modified. The leading industry standard is C2PA Content Credentials. When an AI tool creates or edits a piece of media, C2PA embeds a record of what tool was used and what changes were made.

This is different from the thin labels criticized above. Labels depend on platforms or creators to identify AI content after the fact – provenance is embedded at the point of creation and travels with the file itself.

C2PA adoption by AI content generators is voluntary, and users can remove provenance signals. But when provenance survives, it gives viewers, platforms, and regulators a reliable way to check where content came from.

Some platforms already read provenance signals to apply labels. But even those platforms typically strip the underlying metadata when content is uploaded, breaking the chain of custody. The Washington Post tested a C2PA-tagged deepfake across eight major platforms and found that almost none preserved or displayed the signal.

The fix is narrow and enforceable: policymakers should require major platforms to preserve provenance when it exists and display it consistently to users.

In the EU and UK, a provenance rule could attach to existing measures under the AI Act and Online Safety Act, respectively. In the US, a durable ‘preserve and display provenance’ obligation likely needs legislation.

Provenance will never be universal, but making it durable creates a reliable baseline signal and gives creators a reason to adopt it.

3. Use distribution circuit breakers in high-risk moments

The core problem with AI misinformation is that fake content can spread faster than verification or enforcement can catch up.

Regulators should require platforms to slow the spread of deepfakes in defined high-risk windows, such as the final days before an election. Platforms can reduce amplification for fast-spreading posts and add friction to rapid resharing.

This does not require platforms to detect deepfakes. Instead, platforms would target content that a) lacks provenance and b) matches high-risk distribution patterns – paid political content, high-reach accounts, or posts crossing virality thresholds.

In the EU, the DSA elections toolkit already urges platforms to limit the amplification of deceptive content and ensure labels persist when reshared. In the US, Congress would likely need to set clearer national guardrails, though broad speech protections make this route more contested, as state deepfake laws are already facing First Amendment challenges.

The goal is not perfect truth enforcement, but rather temporary friction that reduces harm at critical times.

Prevention is better than a cure

These three interventions won’t eliminate deepfakes, but together, they could change the environment in which they operate.

AI impersonation would become riskier. Wider provenance adoption would let platforms and users see where content came from. And in high-risk moments like elections, distribution friction would reduce the speed and scale at which misinformation can spread.

Detection and labeling still have a role, but mainly as supporting tools for triage, review, and evidence, not as the primary architecture of prevention. The aim should not be to catch every fake after the damage is done, but to make harmful deepfakes harder to weaponize, easier to trace, and less likely to go viral before anyone can respond.

Disclosure: The views expressed are the author’s own and do not necessarily reflect those of his affiliated institutions. The author has no relevant financial conflicts to disclose.

Muhammad Irfan is a deepfake forensics and cybersecurity researcher, and Lecturer at Wentworth Institute of Technology.