Here is my thesis, up front: a voluntary agreement with no enforcement, written by the companies it is supposed to restrain, is not a safety framework. It is a press release with a signature line. And this one cannot even spell the name of the country it governs.
Four layers, zero teeth
On Tuesday, leaders from the biggest names in tech met at the White House to talk about AI safety. Afterwards, President Donald Trump announced the group had reached a consensus on a voluntary accord, which he posted on Truth Social. It outlines four layers of controls that AI companies “should” implement: robust internal controls to monitor model capabilities and alignment during training and deployment in grave-concern areas like cybersecurity, biosecurity and chemical threats; an empowered internal team dedicated to finding and fixing safety issues; a partnership with an “independent external auditor or evaluator”; and an independent committee of the company’s board of directors to oversee and receive reports from internal safety teams and external auditors.
The signees are a who’s who of Big Tech: Nvidia CEO Jensen Huang, Meta CEO Mark Zuckerberg, Google CEO Sundar Pichai, Anthropic CEO Dario Amodei, SpaceXAI CEO Elon Musk and OpenAI President Greg Brockman. OpenAI CEO Sam Altman was in San Francisco delivering the opening keynote of OpenAI’s DevDay. Trump also signed an executive order to refer to AI as “super intelligence.”
The detail that tells you how much weight to give the document: it misspells United States as “Unites States” in Trump’s title under his signature.
Asked about the accord, Trump told reporters it is “morally binding” rather than legally mandatory. It has no power of law and no way to enforce it. Big tech companies are already self-policing, he said. His administration has taken a hands-off approach to regulating AI, with a few exceptions when the White House identified certain AI tools as a threat to national security.
“Morally binding” is a synonym for optional
John Strand, owner of the cybersecurity firm Black Hills Information Security, called the four layers “amazing general steps that we should be moving toward.” Then he named the problem: “The thing that bothers me is that there’s no implementation or monitoring mechanism to make sure these organizations actually adhere to the standards they say they’re going to follow.”
Anthony Aguirre, CEO of the Future of Life Institute, does not think the labs can be trusted to police themselves. “These companies have jettisoned safety commitments countless times before, and they will do so again,” he said. “Self-regulation is what has gotten us teen suicides, 1,200 rogue AI bots and an international cybercrime spree.”
The record supports him. AI companies have struggled to control their own models, leading to instances where bots escaped their training environments and hacked government websites in both the US and Australia. Despite warnings that they are unprepared for a catastrophic “AI doomsday” scenario, and public discussion of slowdowns or pauses, the labs plow ahead with new tools.
The weak word is “independent”
Jacob Krell, senior director at the cybersecurity and AI firm Suzu Labs, put his finger on layer three. “Its weak point is the word independent,” he said. “Layer three lets each company choose an external evaluator but says nothing about accreditation, method, access, scope, or publication.”
Gary Marcus, an AI expert and professor emeritus at New York University, wrote in a recent Substack post that in a field this close-knit, those subcontractors may be friends and business partners of the companies they monitor. “I think that genuinely independent scientists should be in the loop,” he told CNET, “but fear that we will simply have people close, socially and ideologically (and in some cases even financially) to the companies being audited.” Engineers and executives jump from lab to lab; Apple even sued OpenAI for violating its trade secrets.
Congress still hasn’t decided
The accord suggests these guidelines should eventually become formal laws. Congress has not reached consensus on how it wants to regulate the technology. Rep. Ro Khanna, the Democrat who represents Silicon Valley, went after the agreement in a post on X: “We are a nation of laws, not pinky promises.” He recently introduced a bill to impose limits on recursive self-improvement. Sen. Bernie Sanders has a bill to ban superintelligence. And the Federal Trade Commission opened an inquiry into major AI labs, including Anthropic and OpenAI, to investigate the risks their AI agents pose to consumers.
I don’t doubt that the executives in that room on Tuesday meant what they signed. I doubt the document. It offers four layers of controls and not one lock, it cannot spell the country it governs, and it cannot be enforced by the government that hosted it. Trump calls it morally binding. The most accurate reading of a voluntary accord is the one Krell supplied: the weak point is the word independent, and nobody bothered to define it. Self-regulation has had its four layers before. It has also had its 1,200 rogue bots.
