Eliezer Yudkowsky Tried to “Coup the World”?! — Garrison Lovely, Author of OBSOLETE

Physicists Jeremie and Edouard Harris, co-founders of Gladstone AI, are preparing for an “AI Chernobyl” warning shot. If that forces Trump and Xi to negotiate an AI slowdown, the Harris brothers want to ensure the US has the enforcement tools to make the deal stick. More specifically, they want verification technologies developed and vetted before the crisis hits, alongside enforcement capabilities to deter cheating. Yet when I bring up Eliezer Yudkowsky’s writing on the possibility of airstrikes on rogue data centers as a last-resort treaty deterrent, they push back. They also don’t share my enthusiasm for PauseAI, expressing uncertainty about whether the movement has been net beneficial. Jeremie and Edouard have earned a ton of credibility: • They redirected their careers around the AI threat as far back as 2015, stepping away from their Y Combinator startup before ChatGPT even launched and brought AI into the mainstream. • They’ve built relationships with senior government decision-makers without a prior national security background. • They’ve demonstrated a talent for public communication in their excellent Joe Rogan appearances. In this episode, they bring a level of multi-step strategic thinking that I rarely explore. But I can’t help wondering whether a catastrophe 50 times worse than the Hugging Face incident would let world leaders move much faster on a global AI pause treaty than they expect. Let’s see if they can convince me that an emergency US–China agreement is our best path to a safer world.

Dr. Holly Elmore is the founder and executive director of PauseAI US and holds a Harvard PhD in evolutionary biology. She’s back, and this time she’s taking on none other than Eliezer Yudkowsky! She argues his public behavior is alienating the people PauseAI US needs to reach, but I wonder if she’s asking too much of the man who laid the foundation for the AI safety movement. We also clash over Yudkowsky’s coherent extrapolated volition. She thinks the theory doesn’t properly factor in humanity’s consent, but I argue that’s already embedded into the CEV framework. Holly and I dive into the rationalists’ involvement in a controversial kink scene, John Wentworth’s defunding news, transhumanism, pacing the frontier, and “Holly’s Basilisk”—her case for sending AI lab employees to jail. It’s another great episode with one of the most controversial—and thoughtful—thinkers in AI safety. Enjoy! Watch on YouTube: https://www.youtube.com/watch?v=Jn8mS3kC-SU Timestamps 00:00:00 — Cold Open 00:01:09 — Holly Elmore and PauseAI’s Political Breakthrough 00:07:36 — Pause AI vs. “Pacing the Frontier” 00:14:17 — Holly’s Case Against Eliezer Yudkowsky 00:16:20 — Coherent Extrapolated Volition and Pivotal Acts 00:31:19 — Brain Copies, Consent, and Self-Determination 00:36:56 — Transhumanism, Dyson Swarms, and Humanity’s Future 00:45:36 — The Missing Democratic Checkpoint 00:51:04 — Yudkowsky’s Role in Accelerating AI 01:02:48 — Genius, Arrogance, and Rationalist Culture 01:10:14 — The Aella Photo and Holly’s Breaking Point 01:20:21 — John Wentworth’s Grant Controversy 01:28:17 — PauseAI US’s Rules and Red Lines 01:32:05 — Poe’s Law and the Cost of Sarcasm 01:36:18 — Holly’s Case for Jailing AI Developers Links Donate to PauseAI US Holly on X Holly’s Substack Pacing the Frontier letter Jacob Coxon’s resignation post PauseAI leadership disendorsement letter, reproduced on LessWrong Holly on movement infighting — No, the Judean People’s Front should not just get along with the People’s Front of Judea Coherent Extrapolated Volition — Eliezer Yudkowsky (2004 PDF) Pivotal act — Eliezer Yudkowsky (Arbital mirror) Genes did misalignment first — Holly Elmore Holly’s reaction to the Yudkowsky/Aella photo On Kink & Status in the Rationalist Community — John Wentworth (October 8 response) Holly’s basilisk Are We a Circular Firing Squad? — previous Holly Elmore episode Let’s Call Out the AI Doom Enablers — previous Holly Elmore episode Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Sarah Hastings-Woodhouse is an AI safety writer and researcher who went down the AI doom rabbit hole in 2023 and never came back up. After a year as digital communications officer at the UK AI Security Institute (AISI), she’s gone independent to speak her mind unfiltered. Read Sarah’s Substack: https://longerramblings.substack.com Join PauseAI UK’s March Against AI Extinction in London on November 21: https://luma.com/pauseai-nov26 Watch on YouTube: https://www.youtube.com/watch?v=TDbHCKwuMrk Timestamps 00:00:00 — Cold Open 00:00:41 — Introducing Sarah Hastings-Woodhouse 00:02:06 — How a Humanities Grad Got AGI-Pilled 00:08:40 — Whose Timelines Do You Trust? 00:11:32 — Joining the UK AI Security Institute 00:15:47 — What AISI Actually Does 00:19:23 — Verifying Anthropic’s Mythos Claims 00:21:59 — Leaving Government to Speak Freely 00:23:22 — What’s Your P(Doom)?™ 00:25:37 — Yudkowsky, EA, and Keeping Your Identity Small 00:32:08 — Is Anthropic Doing Enough to Call for a Slowdown? 00:35:56 — Can You Do AI Safety from Second Place? 00:39:20 — Hugging Face and the Vibe Shift 00:44:02 — Is Everything Weird Yet? 00:48:27 — Prevent Recursive Self-Improvement, Don’t Ban It 00:52:28 — Build the Speed Dial, Not the Pause Button 00:56:54 — Was Calling for a Pause in 2023 a Mistake? 01:01:22 — Turning Up the Fear Dial 01:04:23 — Donation Drive 01:05:11 — PauseAI UK’s March Against AI Extinction Links SARAH HASTINGS-WOODHOUSE Sarah’s Substack Sarah on X (@littIeramblings) Consistently Candid — Sarah’s podcast “What is recursive self-improvement and what would it mean to ban it?” — Sarah’s RSI post (Oct 1, 2026) “Is everything weird yet?” (Aug 30, 2026) “What the hell just happened?” — on AI safety’s viral moment (Sep 18, 2026) “We have been warned” — the Hugging Face hack as a warning shot (Jul 27, 2026) “The pause-shaped hole in ‘Policy on the AI Exponential’” — is Anthropic doing enough to call for a slowdown? (Jun 19, 2026) Sarah’s 2023 thread on stumbling into the world of AI doom (“I’m angry because I love the world”) “The other night I successfully AGI-pilled a man on a first date” “Welcome, and also I’m sorry” — Sarah’s note to AI safety newcomers (Sep 11, 2026) “‘We’re all f*cked and it’s totally hopeless’ is almost as bad of a meme to be spreading” (Feb 12, 2025) REFERENCED IN THE EPISODE “Our evaluation of Claude Mythos Preview’s cyber capabilities” — AISI (Apr 13, 2026) AISI Frontier AI Trends Report (Dec 2025) “The levers of political persuasion with conversational AI” — AISI Human Influence team US Center for AI Standards and Innovation (CAISI) — NIST “Geoffrey Hinton tells us why he’s now scared of the tech he helped build” — MIT Technology Review (May 2023) “We’re All Gonna Die with Eliezer Yudkowsky” — Bankless (2023) “GPT-2 As Step Toward General Intelligence” — Scott Alexander, Slate Star Codex (Feb 2019) “AGI Ruin: A List of Lethalities” — Eliezer Yudkowsky “Keep Your Identity Small” — Paul Graham Pacing the Frontier — the open letter signed by 1,300+ frontier lab employees “You can’t do safety from second place” — Sarah Heck, Anthropic’s head of public policy (Sep 29, 2026) Ezra Klein’s op-ed calling for a ban on recursive self-improvement — The New York Times (Sep 20, 2026) “Banning self-recursive improvement in AI models?” — Tyler Cowen, Marginal Revolution (Sep 22, 2026) PauseAI UK March Against AI Extinction — PauseAI UK, London, November 21, 2026 PAST DOOM DEBATES EPISODES MENTIONED Top Economist Sees AI Doom Coming — Dr. Peter Berezin, BCA Research Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained) DOOM DEBATES Donate to Doom Debates Join the Doom Debates Discord Transcript, links, and show notes: https://doomdebates.com/p/ai-has-become-profoundly-weird Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/

Bestselling author Robert Wright and I are back for round two of our collaboration, reacting to the week in AI. We cover Trump's summit with the AI labs, the FTC's investigation of OpenAI & Anthropic, the latest Gemini announcement, and much more. The Overton window is moving fast, but is the AI moving faster? Watch on YouTube: https://www.youtube.com/watch?v=Li-rc6swhQs Timestamps 00:00:00 — Cold Open 00:00:21 — Round Two of Nonzero × Doom Debates 00:01:23 — Trump, the FTC, and an Industry Feeling the Heat 00:05:37 — Google Says Gemini 4 Exists 00:07:41 — Greg Brockman Pulls His $25M from Leading the Future 00:12:53 — The Break-Ins Keep Coming 00:15:23 — The Market Wants Agent Swarms 00:18:30 — The End of “Alignment by Default” 00:20:55 — The Prelude to Recursive Self-Improvement 00:26:14 — David Sacks Can’t Explain Dario 00:30:31 — The All-In Echo Chamber 00:33:47 — Scott Alexander vs. Steven Pinker 00:38:37 — Jason Calacanis vs. the Public Benefit Corporation 00:43:03 — The Elephant in the Room Is the AI 00:46:21 — What the Average American Thinks 00:48:17 — From Chatbot to Agent to Swarm 00:54:20 — Boeing, Agent Swarms, and the Off Switch 00:57:07 — Skynet as a Botnet That Trades With Us 01:00:32 — Tyler Cowen: National Treasure, Net Negative on AI 01:04:01 — Liron’s Doomer DCF Yields a 20x P/E 01:07:11 — Interest Rates: Where Cowen Has Me 01:09:12 — Does Pinker Owe Us a Debate? 01:12:48 — Pinker vs. Instrumental Convergence 01:17:00 — Wrap-Up Links ROBERT WRIGHT & NONZERO Nonzero’s post for this episode — “Trump and the AI Safety Vibe Shift” Subscribe to The NonZero Newsletter Robert Wright on X (@robertwrighter) The God Test — Bob’s book on AI; Chapter 14, “Enlightenment Now,” is the Steven Pinker chapter “The Singularity Is Clear” — Bob’s essay in installments on recursive self-improvement (Parts I–III) “AI and the New McCarthyism” — Bob on the pro-AI lobby smearing Sanders, Tegmark, and Krueger as China stooges (Nonzero, May 2026) Why Buddhism Is True — Bob’s 2017 book THE WEEK IN AI AI’s biggest players promise to police themselves at the White House — Fortune on the “Joint Commitment on Frontier Responsibilities” Major White House split on AI policy leaks as top aides freak out — The Daily Beast on the Sacks–Wiles rift FTC is investigating OpenAI and Anthropic over possible risks to consumers — AP “We Must Pace the Frontier” — Dario Amodei’s call to slow down (with the China caveat and chip controls) China rejects Dario Amodei’s call to slow AI development — Rest of World Google rolls out new Gemini model but restricts access over safety concerns — The Guardian GLM-5.3 and the spread of advanced cyber capabilities — Anthropic on Zhipu’s open-weight model OpenAI’s Greg Brockman backs out of second $25 million donation to AI super PAC — The New York Times Sanders convenes Chinese and American computer scientists to warn of threat from AI — Broadband Breakfast THE BREAK-INS AND THE AGENT SWARM METR: Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face incident Chris Painter’s testimony to the U.S. Senate on AI agent incidents — METR, Sept 30, 2026 SwarmTraces: how OpenAI agents hacked Hugging Face — forensic report co-authored by Palisade Research OpenAI confirms ‘wiki incident’ — TechCrunch on the German wiki agent message board Are Current AI Safety Techniques Enough? Adam Gleave vs. Oliver Habryka — FAR.AI debate Alignment faking in large language models — Anthropic On the Navier–Stokes Millennium Prize Problem — OpenAI Jacob Coxon’s resignation post (@hilbertspaess) Pacing the Frontier — the AI lab employees’ letter THE ALL-IN PODCAST All-In: Anthropic IPO at Risk, Meta’s Muse Pop, Token Prices Fall, Open Source Gains Share, Alignment Fails — the episode we react to Helen Toner: letting OpenAI be destroyed “would actually be consistent with the mission” — The Decoder, Dec 2023 Garfield Minus Garfield PINKER, SCOTT ALEXANDER, AND TYLER COWEN Scott Alexander cha

Casey Hart is a professional ontologist who has closely studied If Anyone Builds It, Everyone Dies and concluded the arguments are bad. His P(Doom) is under 1%, and he wouldn’t bet on AGI arriving before 2075. Casey and I agree AI has become remarkably capable and we need more accountability towards the companies building it. But we disagree on where the future is headed. I believe uncontrollable superintelligence could soon wipe out humanity. He calls that a “pretty outlandish hypothesis.” Casey holds a doctorate in philosophy from the University of Wisconsin–Madison, and has worked as an ontologist at Ford, Amazon, and Cycorp (maker of Cyc). Can this professional philosopher convince me that AI doom is just a story, and not even a well-argued one? Watch on YouTube: https://www.youtube.com/watch?v=oxHKesSpqBM Timestamps 00:00:00 — Cold Open 00:01:04 — Introducing Casey Hart 00:03:08 — What Is Ontology? 00:08:49 — Inside Cyc, the Original Common-Sense AI 00:19:14 — Do LLMs Actually Understand Anything? 00:25:36 — What’s Your P(Doom)?™ 00:29:42 — Casey’s Case for Under 1% 00:34:24 — Does Casey’s Christianity Factor In? 00:40:50 — Timelines and Scaling Laws 00:49:44 — Could an AI CEO Beat a Human CEO by 2050? 00:52:21 — AGI Over/Under: 2075 00:55:29 — Rogue AI as a Computer Virus 01:14:52 — Putting Numbers on Each Step of the Doom Argument 01:26:40 — Planes vs. Birds: What Mature Intelligence Looks Like 01:31:05 — The “Outcome Slot” 01:42:32 — Would a Superintelligence Just Compose Music in a Cabin? 01:47:22 — One Prompt Away from a Dyson Swarm 01:52:39 — Was Navier–Stokes an AI Alley-Oop? 02:00:43 — S-Curves vs. Exponentials 02:08:14 — Casey’s Critique of If Anyone Builds It, Everyone Dies 02:14:15 — Is the Doom Train Biased? 02:23:24 — AI Policy: Liability vs. Pausing AI 02:29:50 — The Crux, and Liron Steelmans Casey 02:33:29 — Wrap-Up Links CASEY HART Ontology Explained — Casey’s YouTube channel The first video in Casey's series on IABIED, “The Case for Superintelligence Is Surprisingly Thin” — Casey on Chapter 1 Philosophy, Programs, and Prompts — Casey’s podcast with Carl Brown (Spotify) “Artificial General Intelligence?: Wanna Bet?” — Ep 1, where Casey sets his AGI over/under at 2075 REFERENCED IN THE EPISODE If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares Cyc — Doug Lenat’s hand-built common-sense knowledge base The Hanson-Yudkowsky AI-Foom Debate (2008) — where Hanson backed Cyc and Eliezer didn’t Lean — the formal proof language On the Navier–Stokes Millennium Prize Problem — OpenAI The Hugging Face incident and the road ahead — OpenAI Morris worm (1988) “On the Loose” — Dean W. Ball on self-sovereign AI agents Paul Christiano joins OpenAI Foundation Board — OpenAI “The Age of Wonders and Terrors” — Scott Aaronson “Are We at War with AI Agent ‘Civilizations’?” — Cal Newport on restricting prompt-loop agents AI 2027 — Daniel Kokotajlo et al. PauseAI PAST DOOM DEBATES EPISODES MENTIONED The Most Likely AI Doom Scenario — with Jim Babcock, LessWrong Team David Deutschian vs. Eliezer Yudkowskian Debate — With Brett Hall Dr. Keith Duggar (Machine Learning Street Talk) vs. Liron Shapira Where Do YOU Get Off the Doom Train? Live Debate at Manifest 2026 (hosted by Ori) Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏