Legal bans on AI models (blacklisting) are necessary to ensure responsible AI development

What's this about?

People disagree about whether laws should ban some AI models to keep AI safe. The key question is whether broad bans work better than strict rules.

What supporters say

• Some AI tools may cause severe harm through lies, unfair choices, scams, privacy loss, cyberattacks, or physical harm.
• The EU AI law bans some harmful uses, such as social scores and certain face scans.
• Once people release a risky AI model, others may copy it, making harm hard to stop.
• Company promises have not stopped many AI failures, so leaders may need to block unsafe systems.

What critics say

• Rules based on each risk can protect people without banning whole AI models.
• Governments may struggle to find and stop banned models online.
• Wide bans could slow helpful new tools in health, school, work, and science.

How to read this

The number of points on each side does not show who is right; strong proof matters more than a long list.

The bottom line

The claim remains unproven. The evidence more clearly supports strict rules and bans for certain very risky uses, not broad bans on AI models.

The claim that responsible AI development requires legal blacklisting of models remains unproven. The evidence supports strong regulation and, in some cases, outright bans—but it more clearly supports targeted restrictions than a general ban on AI models.

The case for

Some AI capabilities may be too dangerous to permit, even when ordinary safety measures are available. AI systems can contribute to manipulation, discrimination, privacy violations, fraud, disinformation, cyberattacks and physical harm. Guidance on high-consequence risks therefore supports staged deployment, strict controls and, in some cases, prohibition of specific capabilities 1.

The European Union’s AI Act puts this principle into law. It bans defined practices including certain manipulative or exploitative systems, social scoring and some forms of biometric identification. This points to a narrower but defensible argument: when a capability creates severe, difficult-to-reverse risks and cannot meet basic safety or accountability requirements, legal prohibition may offer more protection than relying on remedies after harm occurs.

A second argument is that voluntary safeguards have not stopped repeated AI failures. The AI Incident Database records numerous real-world problems, while the AI Index reports rising documented incidents as capabilities expand and standardized safety testing remains limited. Researchers have also highlighted scalable bias, misinformation, environmental damage and poor documentation in large language models (see Figure 1; see Figure 3). These findings support mandatory oversight and leave room for blacklisting systems that cannot meet safety requirements 2.

There is also concern that some high-risk systems may become difficult to control once they are widely released. If model weights or capabilities spread beyond the original developer, later regulation may be unable to undo the damage. That possibility supports strict controls before release and, in exceptional cases, a ban 3.

The case against

The strongest objection is that risk-based regulation is a less blunt alternative. The EU framework bans certain practices but places most other systems into different risk categories, with obligations such as impact assessments, testing, documentation, human oversight, transparency, monitoring, incident reporting and accountability. Guidance from UNESCO, NIST, the OECD and British authorities similarly emphasizes controls across the system’s life cycle rather than a single blacklist 4.

This approach is intended to manage serious risks while preserving useful experimentation. Research on foundation models points to benefits in productivity, science and downstream applications. OECD analysis also identifies possible gains in productivity and innovation, while the EU supports regulatory sandboxes and differentiated requirements. Still, the evidence does not provide a reliable estimate of how much broad blacklisting would reduce innovation, or how its costs would be shared among companies and users 6.

Enforcement is another major problem. Once an open model’s weights have been released, they can be copied or modified. Restrictions may simply move development to other countries or less transparent operators, while lawful users lose access without eliminating the underlying capability 5. For that reason, policy researchers often favor evaluations, access controls, monitoring and accountability alongside deployment restrictions.

The available evidence also cannot show that blacklisting caused fewer harms than other tools would have. Incident records demonstrate that harms occur, but they do not identify which regulatory measures prevented or caused them. There is limited evidence about enforcement after capabilities spread, substitution by foreign providers, or the point at which a system becomes uncontrollable enough to justify prohibition.

The bottom line

The evidence favors targeted bans, not general model blacklisting—and only with low confidence that broad bans are necessary. Legal prohibition is defensible for narrowly defined uses or capabilities that pose severe risks and cannot meet safety requirements. But the case for banning AI models as a class is much weaker than the case for layered controls aimed at uses, deployments, developers and access.

The central uncertainty is not whether AI can cause serious harm. It can. The unanswered question is whether banning particular models produces better overall safety outcomes than enforceable audits, standards, liability rules, monitoring and deployment restrictions. Current research does not yet make that comparison robustly, so it does not establish that broad model bans are generally required for responsible AI development.