Show Notes
Join me for round 2 with Dr. Peter Park, an AI Existential Safety Postdoctoral Fellow working with Dr. Max Tegmark at MIT. Dr. Park was a cofounder of StakeOut.AI, a non-profit focused on making AI go well for humans, along with Harry Luk and one other individual, whose name has been removed due to requirements of her current position. In addition to the normal links, I wanted to include the links to the petitions that Dr. Park mentions during the podcast. Note that the nonprofit which began these petitions, StakeOut.AI, has been dissolved. Right AI Laws, to Right Our Future: Support Artificial Intelligence Safety Regulations Now Is Deepfake Illegal? Not Yet! Ban Deepfakes to Protect Your Family & Demand Deepfake Laws Ban Superintelligence: Stop AI-Driven Human Extinction Risk
00:00:54 - Intro 00:02:34 - Battleground 1: Copyright 00:06:28 - Battleground 2: Moral Critique of AI Collaborationists 00:08:15 - Rich Sutton 00:20:41 - OpenAI Drama 00:34:28 - Battleground 3: Contract Negotiations for AI Ban Clauses 00:37:57 - Tesla, Autopilot, and FSD 00:40:02 - Recycling 00:47:40 - Battleground 4: New Laws and Policies 00:50:00 - Battleground 5: Whistleblower Protections 00:53:07 - Whistleblowing on Microsoft 00:54:43 - Andrej Karpathy & Exercises in Empathy 01:05:57 - Outro Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. StakeOut.AI The Times Sues OpenAI and Microsoft Over A.I. Use of Copyrighted Work Susman Godfrey LLP Rich SuttonReinforcement Learning: An Introduction (textbook) AI Succession (presentation by Rich Sutton) The Alberta Plan for AI Research Moore's LawThe Future of Integrated Electronics (original paper) Computer History Museum's entry on Moore's Law Stochastic gradient descent (SGD) on Wikipedia OpenAI DramaMax Read's Substack post Zvi Mowshowitz's Substack series, in order of postingOpenAI: Facts from a Weekend OpenAI: The Battle of the Board OpenAI: Altman Returns OpenAI: Leaks Confirm the Story ← best singular post in the series OpenAI: The Board Expands Official OpenAI announcement WGA on Wikipedia SAG-AFTRA on Wikipedia Tesla's False AdvertisingTesla's response to the DMV's false-advertising allegations: What took so long? Tesla Tells California DMV that FSD Is Not Capable of Autonomous Driving What to Call Full Self-Driving When It Isn't Full Self-Driving? Tesla fired an employee after he posted driverless tech reviews on YouTube Tesla's page on Autopilot and Full Self-Driving RecyclingBoulder County Recycling Center Stockpiles Accurately Sorted Recyclable Materials Out of sight, out of mind Boulder Eco-Cycle Recycling Guidelines Divide-and-Conquer Dynamics in AI-Driven Disempowerment Microsoft WhistleblowerWhistleblowers call out AI's flaws Shane's LinkedIn postLetters sent by Jones Karpathy announces departure from OpenAI
More from Into AI Safety
Etienne Brisson and Allan Brooks, founder and program director of The Human Line Project, sat down with me to talk about AI-induced psychological harm, starting with their own experiences. Allan spent a month spiraling with ChatGPT, ending with him warning national security agencies about an encryption break that never happened. Etienne's uncle became convinced his chatbot was sentient and was hospitalized still messaging it from the back of a police car.
Dr. Evan Miyazono joins Jacob to talk about his path from quantum networking and Protocol Labs to founding Atlas Computing. They discuss why Atlas moved away from formal methods research and toward what Evan calls “field strategy”: identifying important problems, figuring out what interventions could actually address them, and finding the people best positioned to make them happen. The conversation also covers the resilience gap map, coordination failures, AI safety, "value alignment," and the question of how to decide what work is worth doing when there’s no obvious organization responsible for it. Check out the video version of this podcast on the Kairos.fm YouTube channel, or the extended version on Patreon! Chapters
What if the safest AI models weren't built by adding guardrails after training, but by shaping what gets learned in the first place? Ethan Roland, senior alignment researcher at AE Studio and first author on an ICML 2026 spotlight paper, joins Jacob to talk about gradient routing, a technique that routes dangerous capabilities into isolated parts of a model's architecture where they can be locked or removed entirely. They get into the absorption effect, KYC-style access control frameworks, and what it would actually take for frontier labs to adopt this kind of work before it's needed rather than after. Chapters
Today's episode does double duty as an interview and an announcement. Joe Williams, host of the new Kairos.fm show "Beyond the Paycheck: Reclaiming the Case for UBI in the Age of AI," joins Jacob to talk about his background as a freelance translator and how AI quietly dismantled his livelihood in 2025. From there the conversation expands into whether this moment is really different from past waves of automation, who exactly makes up the tech billionaire class, and why you should probably raise an eyebrow when someone like Elon Musk says he supports UBI. Go check out Reclaiming UBI! Chapters
Zainab Majid, co-founder of Asymmetric Security, joins Jacob for a conversation on the intersection between AI Safety and cybersecurity, as well as the future of digital forensics. Drawing from years of incident response work, she explains how cyber attacks actually unfold, why AI is changing both offense and defense, and how her team is building AI-native tools to investigate breaches faster and more effectively. Other topics explored in this episode include trust in the AI/cybersecurity industries, the realities behind cybersecurity hype, and the challenge of keeping humans meaningfully involved as these systems become more capable. Zainab also gives practical, actionable advice on how you can protect yourself. If you're interested in over 30 minutes of additional content, head on over to the Kairos.fm Patreon where you can become a subscriber for just $2 per month, which helps make this whole podcasting thing a bit more sustainable. Chapters