As countries come together on AI, we urge limiting recursive self-improvement

A statement from AI governance experts, UNGA 2026

By CeSIA, The Future Society, and AI Safety Connect, during the UN General Assembly.

For more than a decade, researchers have warned that advanced AI could escape human control, and that is not the only risk it poses. Governments must act swiftly to prevent major AI-related crises such as threats to national and international security, large-scale manipulation of individuals, including children, systematic human rights violations, and other systemic risks. The Global Call for AI Red Lines, launched at the UN General Assembly a year ago, asked governments to agree on red lines by the end of 2026. This week, a large coalition of countries called for AI to be kept under human control. We welcome that call and outline the concrete requirements for making their call for control a reality.

What we ask

Limit recursive self-improvement (RSI) until it can be done safely. Addressing the risks above takes time, and AI already advances faster than institutions can respond. AI that improves itself would only dangerously widen that gap. Frontier AI companies already recognize and warn about the dangers of crossing this RSI threshold. UN High Commissioner for Human Rights Volker Türk already asked companies to limit it. But today, the companies set, measure, and judge their own limits. Governments should use those to establish a common, independently verified threshold. This is necessary to robustly prevent other systemic risks related to AI.

To make this ambition a reality and prevent systemic risks from advanced frontier AI capabilities, governments must urgently come together to:

1. Clarify liability and give an independent authority the power to enforce regulation.

This authority would require developers to demonstrate that their most capable systems adhere to safety requirements, including for internal deployment.

2. Give real access to independent auditors.

Auditors, not companies, should set the scope of tests and incident investigations and must be able to publish their findings independently. The Minimum Conditions for Embedding Evaluators signed by more than 200 experts show how.

3. Establish clear mechanisms for crisis prevention and management.

Companies must report and investigate serious incidents swiftly, thoroughly, and publicly. Governments must create the capacity for an international response, including appropriate shutdown mechanisms. Standards for how and when to report incidents should be established as soon as possible.

The signatories see these priorities as the most urgent; this list is not exhaustive.

Finally, we call on more governments to join the coalition of countries calling for control of frontier AI models, and on all of them to open negotiations on international thresholds and on an authority capable of verifying them now, with independent scientists and civil society at the table.