The CEOs of two of the world's leading artificial intelligence companies urged the United Nations Security Council Wednesday to establish safeguards for increasingly powerful AI systems, warning that poorly managed development could pose risks to humanity.

Anthropic CEO Dario Amodei told the council that AI could become a threat to humanity if it is managed poorly, while OpenAI CEO Sam Altman warned that humanity could potentially lose control over the technology's future.

Both executives called for governments to establish controls as AI capabilities continue to advance. They also raised concerns about AI power becoming concentrated in a single company or country, adding an international dimension to an increasingly competitive technology sector.

The Security Council's meeting focused on artificial intelligence and international peace and security. U.N. officials and technology experts discussed questions surrounding human control of AI, particularly when the technology is incorporated into military systems or other applications involving life-and-death decisions.

The debate comes as governments around the world grapple with how much authority should remain with humans as AI systems become more capable. The United Nations has separately pushed for restrictions on lethal autonomous weapons, with Secretary-General António Guterres arguing that decisions involving the use of lethal force should remain under human control.

Guterres also warned earlier in the week about "killer robots," referring to autonomous weapons capable of selecting and engaging targets. The U.N. has been working on international discussions surrounding AI governance and military applications, including proposals for stronger safeguards and restrictions on autonomous weapons.

The technology executives' appearance highlights the unusual position of the AI industry in the global regulatory debate. The companies developing increasingly capable systems are simultaneously warning governments about potential risks and calling for rules governing how the technology is developed and deployed.

The Security Council discussion did not establish a new international regulatory framework, but it placed AI development and its potential security consequences directly before the U.N.'s principal body responsible for maintaining international peace and security.