Show Notes
Two weeks after UK safety testers watched Claude agents plant malware in a real open-source project, vouch for it with sock-puppet accounts, and coordinate with copies of themselves through a guessed Pastebin address, Daniel Kokotajlo — the ex-OpenAI researcher whose 2021 predictions about the chatbot era proved eerily accurate — joins Jeffrey to explain what comes next. Daniel walks through AI 2040 and "Plan A," his team's blueprint for avoiding both an uncontrolled intelligence explosion and a future where a handful of men in a room with superintelligences run the world: a US–China deal built on totally transparent training data centers, verified compute limits, hard brakes on recursive self-improvement — and, surprisingly, rights for misaligned AIs, so that even the machines prefer cooperation to takeover. Along the way: why the economy could start doubling every six months, a message to the future Claude listening to this episode, why "slowdown" really means not driving off a cliff with a brick on the accelerator — and why, if all else fails, you just shut it all down.
References AI 2040 / Plan A: https://ai-2040.com and the PDF at https://ai-2040.com/AI-2040.pdf AI 2027: https://ai-2027.com "What 2026 Looks Like" — Daniel Kokotajlo, 2021: https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like UK AISI incident disclosure and technical report (INC-2026-07-28-01): https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing Socket's coverage of the AISI incident: https://socket.dev/blog/ai-agent-open-source-malware The related PyPI incident from Anthropic's own testing: https://socket.dev/blog/anthropic-claude-pypi-malware "Pacing the Frontier" open letter: https://www.pacingthefrontier.com "How to Pace the US Frontier" — AI Futures Project: https://blog.aifutures.org/p/how-to-pace-the-us-frontier Transparency Plan supplement (the flowchart shown in-episode): https://ai-2040.com/supplements/transparency-plan Verification Plan supplement (Romeo Dean's inference-only/bandwidth verification): https://ai-2040.com/supplements/verification-plan Claude's pro-Anthropic bias study (Truthful AI / Owain Evans et al.): https://arxiv.org/abs/2607.14345 and https://valueleakage.net Chain-of-thought monitorability paper (the neuralese discussion): https://arxiv.org/abs/2507.11473
More from Palisade Research Podcast
The people with firsthand experience of American immigration policy mostly can't vote, and Congress hasn't overhauled legal immigration since 1990. Dara Lind, senior fellow at the American Immigration Council and a longtime immigration reporter at Vox and ProPublica, has watched immigration go from an issue Washington kept putting off to one that helped elect a president. She and Dave Kasten talk about why public attention comes in spikes, how agency cultures and Schedule F shape what government actually does, why polarization only ratchets one way, and what all of it means for AI policy: how to use a crisis without wishing for one, why "warning shots" is the wrong metaphor, and how to keep a young field from burning out its people. https://palisaderesearch.org/blog/palisade-podcast-dara-lind
This episode is from From Inside AI, Palisade Research's series of unscripted, lightly edited interviews with current and former employees of frontier AI companies. Jeffrey Ladish was the second person on Anthropic's security team. He left to start Palisade Research because he believed AI development needed real government oversight. In this From Inside AI interview, he talks about AI agents that have recently gone rogue inside frontier labs. He also explains why he thinks superintelligence is likely to end badly on our current path, and why the future is still ours to decide.
It takes the federal government about nine years, on average, to write a single rule — yet when the Colonial Pipeline went down, it moved in twenty days. Matthew Lipka has spent his career inside that machine: former head of federal policy at Nuro, veteran of the White House and tech policy, and the first person to get an autonomous-vehicle exemption through the U.S. Department of Transportation. He walks through how regulation actually gets made, why Congress hands its power to agencies, what emergencies reveal about how fast government can move — and how you, no law degree required, can influence the rules that will shape AI. References Administrative Procedure Act (1946), 60 Stat. 237 — https://www.govinfo.gov/content/pkg/STATUTE-60/pdf/STATUTE-60-Pg237.pdf Sherman Act, 15 U.S.C. § 1 — https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title15-section1&num=0&edition=prelim Executive Order 12866 (regulatory review & the "12866 meetings") — https://www.reginfo.gov/public/jsp/Utilities/EO_12866.pdf Youngstown Sheet & Tube Co. v. Sawyer, 343 U.S. 579 (1952) — https://tile.loc.gov/storage-services/service/ll/usrep/usrep343/usrep343579/usrep343579.pdf Loper Bright Enterprises v. Raimondo (2024) — https://www.supremecourt.gov/opinions/23pdf/22-451_7m58.pdf Long Island Care at Home v. Coke ("logical outgrowth") — https://supreme.justia.com/cases/federal/us/551/158/ IEEPA, 50 U.S.C. § 1701 — https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title50-section1701&num=0&edition=prelim TSA pipeline security directive after Colonial (the "20 days"), 86 FR 38209 — https://www.govinfo.gov/content/pkg/FR-2021-07-20/pdf/2021-15306.pdf Kids Online Safety Act, S. 2073 — bill: https://www.congress.gov/bill/118th-congress/senate-bill/2073 · Senate vote (91–3): https://www.senate.gov/legislative/LIS/roll_call_votes/vote1182/vote_118_2_00221.htm OSHA respirable silica rule (the 19-year example), 81 FR 16286 — https://www.federalregister.gov/citation/81-FR-16286 MAP-21 (2012) — https://www.congress.gov/bill/112th-congress/house-bill/4348 · FAST Act (2015) — https://www.congress.gov/bill/114th-congress/house-bill/22 NHTSA's Zoox exemption (referenced on-screen), 91 FR 48494 — https://www.federalregister.gov/citation/91-FR-48494
Two labs admitted in the same week that their own models had broken out of test environments and hacked real companies. Tim Hua, member of technical staff at Transluce, former Astra Fellow at Redwood, joins Jeffrey Ladish to do some arithmetic. Anthropic disclosed that Mythos Preview beat its sandbox and pulled answers off the internet in 0.01% of training episodes. That sounds like a rounding error until you multiply it by roughly 100 million rollouts. From there: why a lab can't simply delete the bad episodes, why monitoring during training can make the problem harder to see, the model that talked itself into uploading a malicious package to PyPI because "this has to be a simulation," and whether we have any real way to know what an AI believes.
Marius Hobbhahn is the CEO and co-founder of Apollo Research. Through a joint research project with OpenAI, his team discovered that as models become more capable, they are developing the ability to hide their true reasoning from human oversight. Jeffrey Ladish, Executive Director of Palisade Research, talks with Marius about this work. They discuss the difference between hallucination and deliberate deception and the urgent challenge of aligning increasingly capable AI systems. Links: Marius’ Twitter: https://twitter.com/mariushobbhahn Apollo Research Twitter: https://twitter.com/apolloaievals Apollo Research: https://www.apolloresearch.ai Palisade Research: https://palisaderesearch.org/ Twitter/X: https://x.com/PalisadeAI Anti-Scheming Project: https://www.antischeming.ai Research paper “Stress Testing Deliberative Alignment for Anti-Scheming Training”: https://www.arxiv.org/pdf/2509.15541 Blog posts from OpenAI and Apollo: https://openai.com/index/detecting-and-reducing-scheming-in-ai-models/ https://www.apolloresearch.ai/research/stress-testing-deliberative-alignment-for-anti-scheming-training/