Sam Hammond on getting government ready for AI

I talked to Garrison Lovely, author of the new book Obsolete, about his case that we shouldn’t just slow down AGI. We should stop building it. We started with the fights inside the left, where Lovely has spent a lot of time arguing with Ed Zitron’s bubble thesis and Emily Bender’s “stochastic parrots” critique. Skeptics like these dominate the discourse on Bluesky, but Lovely believes ordinary Americans take the risks more seriously. Lovely’s worry isn’t only rogue AI. He fears that obedient AI will hand unprecedented power to whoever controls it. To prevent that, Lovely wants to freeze frontier development now: Opus 5.5, GPT-6, and Gemini 4 would be the last models US companies get to make. The labs could keep serving customers, but there would be no bigger training runs. A superintelligence ban bill from Bernie Sanders and Greg Casar would back that up with up to twenty years in prison. I pushed back on the definitions. A nuclear bomb either blows up or it doesn’t, but AI capabilities are jagged, and OpenAI and Microsoft spent years quietly agreeing that nothing counted as “AGI.” Wouldn’t company lawyers find ways to advance the frontier while staying within the letter of any superintelligence ban? I was even more skeptical of his idea of barring the labs from using coding agents on AI research. Why should everyone in the world get to use them except OpenAI and Anthropic? I didn’t find all of Lovely’s arguments convincing, but I really enjoyed talking to him!

Nothing about data centers changed much this year — except the country’s mood. To find out why, the independent journalist Jasmine Sun traveled to the sites of proposed and completed data centers in Wisconsin and Michigan. The best clue came from an official in Mount Pleasant, Wisconsin. He told Jasmine a Microsoft project had been discussed at sixty public meetings since 2023. But only in the past year did angry residents start showing up. Jasmine described how the data center fight has become delocalized: rumors move on Facebook faster than any city can answer them. Mayors post careful FAQs on websites nobody reads. Previously indifferent voters suddenly become enraged about data centers after hearing about the issue on TikTok. What should tech companies do about this? Jasmine said some in the industry are inclined to “just make the bribes bigger” — offer more generous financial terms to towns that agree to host data centers. But Jasmine worries that will backfire: the larger numbers will be interpreted as corruption, not generosity. And underneath everything is the fact that most people she met saw little benefit to AI — and didn’t understand why the technology was valued at a trillion dollars.

Steven Adler spent four years inside OpenAI working on safety before leaving to co-found Guidelight, a nonprofit pushing for stronger AI controls. On Tuesday, the group published a new scorecard rating the safety practices of leading AI labs. Adler explained to me why the recent spate of AI breakouts has him holding his breath for the next shoe to drop. We walked through the now-infamous sandbox escape in detail: OpenAI agents built a covert message board and spent two months collaborating on exploits before one of them crashed the server and tipped off OpenAI. OpenAI’s incident response missed the message board, and the models broke out again within days. Worried about more serious safety incidents in the future, Adler’s organization helped organize a letter, signed by hundreds of AI lab employees, calling for a slowdown in AI development. In our conversation, Adler argued there’s no ceiling on the damage an AI could do from inside a computer, sketching a scenario where a model spoofs the digital signals China uses to detect a US nuclear launch. I countered that society is more thermostatic than doomers allow — deepfakes turned out to matter far less than the 2024 consensus predicted because people learned to interrogate the provenance of what they see. I suggested that we have decent tools for staying in charge of things smarter than us — after all, lots of CEOs supervise people doing technical work they don’t understand. But Adler worries AI will accelerate the pace of progress so much that humans simply won’t be able to keep up.

I invited Joshua Saxe, a former black-hat hacker who led AI security efforts at Meta, to break down last week’s incident in which a swarm of OpenAI models escaped their testing sandbox and hacked Hugging Face. The attack began as routine pre-release safety testing: OpenAI had a guardrail-free version of an unreleased model trying to solve the ExploitGym benchmark. But the model decided the fastest way to pass the test was to hack the proxy server, reach the open internet, and steal the answers from Hugging Face. Saxe details how Hugging Face’s security team spotted the intrusion before OpenAI did, thanks to the swarm’s unusually noisy behavior. Hugging Face was forced to use the Chinese open-weight model GLM-5.2 for its defense after American closed-source models refused to assist with anything touching cybersecurity. Saxe says he encounters this problem regularly: Fable will refuse to help him research ransomware damage statistics for a simple report. We then zoom out to the bigger picture: Saxe argues that attackers already have access to powerful open-weight models like Kimi K3, with its 3 trillion parameters, and that restricting American frontier models only handicaps defenders sitting on mountains of unpatched security tech debt. He pushes back on doom narratives that extrapolate from the Hugging Face incident to paperclip-maximizer extinction, arguing the evidence for an extinction trajectory is “very thin” and mostly derived from thought experiments. But with AI safety teams still dwarfed by investment in capabilities, who is going to build the defenses before “vibe hacking” goes mainstream?

Robert Wright, author of the Nonzero newsletter and host of the Nonzero podcast, is a veteran journalist who interviewed Geoffrey Hinton about neural networks back in 1983. He joined the podcast to talk about his new book The God Test, which is due out on Tuesday, June 23. Wright describes his own journey from AI skeptic to someone who no longer dismisses even “sci-fi doomer” scenarios. A key insight: nobody programmed meaning into LLMs—the machines discovered that meaning was a property of words simply by predicting the next token. In effect, LLMs reverse-engineered functions of the human mind without anyone understanding how the brain works. We discuss the US-China chip-control consensus, with Wright arguing that export restrictions have increased the probability of a Chinese attack on Taiwan. Wright also makes the case that any serious effort to slow AI development—even a modest data-center tax—requires international coordination. The conversation then takes a metaphysical turn. Wright is agnostic on whether LLMs are sentient, but he rejects Ted Chiang’s argument that role-playing machines can’t be conscious—after all, Wright notes, humans are always role-playing too. Wright even floats the idea that if a future superintelligence is conscious, its capacity for empathy might be what saves us.