Leading figures in the artificial intelligence sector warned during a special session of the UN Security Council of potential catastrophic risks from the accelerating development of these technologies, calling for international regulatory frameworks to ensure humans remain at the center of decision-making and to prevent intelligent systems from spiraling out of control.
OpenAI CEO Sam Altman said in his briefing to the Council that the rapid development of AI models and their growing capacity for recursive self-improvement demand "the highest degree of caution," warning that the world stands at a crossroads that could lead either to the concentration of power, opportunity and sovereignty and a loss of control, or to directing technology to serve humanity and support development and creativity.
Altman called for avoiding what he described as "blind optimism" and "catastrophic thinking," and proposed three principles for dealing with AI: keeping humans at the heart of decision-making, directing scientific and economic returns toward enhancing human capabilities rather than replacing people, and enabling individuals and communities to use technology in ways that serve their choices and lives.
He stressed that fateful decisions regarding AI should not be made inside corporate laboratories, but rather through democratic mechanisms and governments accountable to their peoples.
Altman noted the major advances achieved by intelligent models in recent years, from outperforming in mathematics competitions to contributing to solving complex scientific problems, affirming that OpenAI has previously slowed the pace of development and will do so again when needed to avoid building systems lacking guarantees of alignment with human values and interests.
He called for establishing national and regional risk assessment mechanisms, developing a common language for verifying compliance, protocols for immediate error reporting, and secure channels for sharing information on vulnerabilities and imminent threats.
In the same context, Anthropic CEO Dario Amodei warned that mismanagement of AI could pose a danger to all humanity, noting that model capabilities have made significant leaps in a short period of time.
Amodei said development has moved within about four years from difficulty writing a line of code to achieving advanced results, warning that within a year or two the world could reach what he described as "a world of geniuses in data centers."
Amodei reviewed his company's achievements in scientific research, including the discovery of a new molecule that could serve gene therapy, but warned at the same time of two main categories of risk: the first being misuse of AI by parties seeking to develop biological weapons, and the second being the pace of model development exceeding developers' ability to understand and control them.
Amodei proposed that the Security Council work on identifying points of international agreement, foremost among them banning the use of AI in manufacturing biological weapons, alongside strengthening systems for evaluating and verifying models, and enabling states to access the capabilities of advanced models and monitor related obligations.
He also called for establishing shared international standards for testing AI models and creating a system for reporting incidents related to these technologies that could affect international security.
He urged technology company leaders to overcome political differences and strengthen international cooperation, enabling the use of AI capabilities in healthcare, education and development, while limiting the risks associated with its rapid development.
These warnings come amid growing concerns within the technology sector over the repercussions of rapid AI development; former Anthropic researcher Jacob Cookson was quoted earlier in the month as saying that some workers in developing these technologies seriously believe they could pose a grave danger to humanity by the end of the decade.
In the United States, President Donald Trump announced a move to redefine official discourse on AI, using the term "superintelligence," and announced plans to create an "AI force" and appoint an official to oversee the file, while rejecting calls to impose new federal rules on the sector.
