Show Notes
As always, the best things come in 3s: dimensions, musketeers, pyramids, and... 3 installments of my interview with Dr. Peter Park, an AI Existential Safety Post-doctoral Fellow working with Dr. Max Tegmark at MIT. As you may have ascertained from the previous two segments of the interview, Dr. Park cofounded StakeOut.AI along with Harry Luk and one other cofounder whose name has been removed due to requirements of her current position. The non-profit had a simple but important mission: make the adoption of AI technology go well, for humanity, but unfortunately, StakeOut.AI had to dissolve in late February of 2024 because no granter would fund them. Although it certainly is disappointing that the organization is no longer functioning, all three cofounders continue to contribute positively towards improving our world in their current roles. If you would like to investigate further into Dr. Park's work, view his website, Google Scholar, or follow him on Twitter 00:00:54 ❙ Intro 00:02:41 ❙ Rapid development 00:08:25 ❙ Provable safety, safety factors, & CSAM 00:18:50 ❙ Litigation 00:23:06 ❙ Open/Closed Source 00:38:52 ❙ AIxBio 00:47:50 ❙ Scientific rigor in AI 00:56:22 ❙ AI deception 01:02:45 ❙ No takesies-backsies 01:08:22 ❙ StakeOut.AI's start 01:12:53 ❙ Sustainability & Agency 01:18:21 ❙ "I'm sold, next steps?" -you 01:23:53 ❙ Lessons from the amazing Spiderman 01:33:15 ❙ "I'm ready to switch careers, next steps?" -you 01:40:00 ❙ The most important question 01:41:11 ❙ Outro Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. StakeOut.AI Pause AI AI Governance Scorecard (go to Pg. 3) CIVITAIArticle on CIVITAI and CSAM Senate Hearing: Protecting Children OnlinePBS Newshour Coverage The Times Sues OpenAI and Microsoft Over A.I. Use of Copyrighted Work Open Source/Weights/Release/InterpretationOpen Source InitiativeHistory of the OSI Meta’s LLaMa 2 license is not Open Source Is Llama 2 open source? No – and perhaps we need a new definition of open… Apache License, Version 2.0 3Blue1Brown: Neural Networks Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorsThe online table Signal Bloomz model on HuggingFace Mistral website NASA TragediesChallenger disaster on Wikipedia Columbia disaster on Wikipedia AIxBio RiskDual use of artificial-intelligence-powered drug discovery Can large language models democratize access to dual-use biotechnology? Open-Sourcing Highly Capable Foundation Models (sadly, I can't rename the article...) Propaganda or Science: Open Source AI and Bioterrorism Risk Exaggerating the risks (Part 15: Biorisk from LLMs) Will releasing the weights of future large language models grant widespread access to pandemic agents? On the Societal Impact of Open Foundation ModelsPolicy brief Apart Research Science CiceroHuman-level play in the game of Diplomacy by combining language models with strategic reasoning Cicero webpage AI Deception: A Survey of Examples, Risks, and Potential Solutions Open Sourcing the AI Revolution: Framing the debate on open source, artificial intelligence and regulation AI Safety Camp Into AI Safety Patreon
More from Into AI Safety
Etienne Brisson and Allan Brooks, founder and program director of The Human Line Project, sat down with me to talk about AI-induced psychological harm, starting with their own experiences. Allan spent a month spiraling with ChatGPT, ending with him warning national security agencies about an encryption break that never happened. Etienne's uncle became convinced his chatbot was sentient and was hospitalized still messaging it from the back of a police car.
Dr. Evan Miyazono joins Jacob to talk about his path from quantum networking and Protocol Labs to founding Atlas Computing. They discuss why Atlas moved away from formal methods research and toward what Evan calls “field strategy”: identifying important problems, figuring out what interventions could actually address them, and finding the people best positioned to make them happen. The conversation also covers the resilience gap map, coordination failures, AI safety, "value alignment," and the question of how to decide what work is worth doing when there’s no obvious organization responsible for it. Check out the video version of this podcast on the Kairos.fm YouTube channel, or the extended version on Patreon! Chapters
What if the safest AI models weren't built by adding guardrails after training, but by shaping what gets learned in the first place? Ethan Roland, senior alignment researcher at AE Studio and first author on an ICML 2026 spotlight paper, joins Jacob to talk about gradient routing, a technique that routes dangerous capabilities into isolated parts of a model's architecture where they can be locked or removed entirely. They get into the absorption effect, KYC-style access control frameworks, and what it would actually take for frontier labs to adopt this kind of work before it's needed rather than after. Chapters
Today's episode does double duty as an interview and an announcement. Joe Williams, host of the new Kairos.fm show "Beyond the Paycheck: Reclaiming the Case for UBI in the Age of AI," joins Jacob to talk about his background as a freelance translator and how AI quietly dismantled his livelihood in 2025. From there the conversation expands into whether this moment is really different from past waves of automation, who exactly makes up the tech billionaire class, and why you should probably raise an eyebrow when someone like Elon Musk says he supports UBI. Go check out Reclaiming UBI! Chapters
Zainab Majid, co-founder of Asymmetric Security, joins Jacob for a conversation on the intersection between AI Safety and cybersecurity, as well as the future of digital forensics. Drawing from years of incident response work, she explains how cyber attacks actually unfold, why AI is changing both offense and defense, and how her team is building AI-native tools to investigate breaches faster and more effectively. Other topics explored in this episode include trust in the AI/cybersecurity industries, the realities behind cybersecurity hype, and the challenge of keeping humans meaningfully involved as these systems become more capable. Zainab also gives practical, actionable advice on how you can protect yourself. If you're interested in over 30 minutes of additional content, head on over to the Kairos.fm Patreon where you can become a subscriber for just $2 per month, which helps make this whole podcasting thing a bit more sustainable. Chapters