
Show Notes
What does it actually take to build a successful AI safety organization? I'm joined by Dr. Ryan Kidd, who has co-led MATS from a small pilot program to one of the field's premier talent pipelines. In this episode, he reveals the low-hanging fruit in AI safety field-building that most people are missing: the amplifier archetype. I pushed Ryan on some hard questions, from balancing funder priorities and research independence, to building a robust selection process for both mentors and participants. Whether you're considering a career pivot into AI safety or already working in the field, this conversation offers practical advice on how to actually make an impact. Chapters
(00:00) - - Intro
(08:16) - - Building MATS Post-FTX & Summer of Love
(13:09) - - Balancing Funder Priorities and Research Independence
(19:44) - - The MATS Selection Process
(33:15) - - Talent Archetypes in AI Safety
(50:22) - - Comparative Advantage and Career Capital in AI Safety
(01:04:35) - - Building the AI Safety Ecosystem
(01:15:28) - - What Makes a Great AI Safety Amplifier
(01:21:44) - - Lightning Round Questions
(01:30:30) - - Final Thoughts & Outro
LinksMATS Ryan's Writing LessWrong post - Talent needs of technical AI safety teams LessWrong post - AI safety undervalues founders LessWrong comment - Comment permalink with 2025 MATS program details LessWrong post - Talk: AI Safety Fieldbuilding at MATS LessWrong post - MATS Mentor Selection LessWrong post - Why I funded PIBBSS EA Forum post - How MATS addresses mass movement building concerns FTX Funding of AI Safety LessWrong blogpost - An Overview of the AI Safety Funding Situation Fortune article - Why Sam Bankman-Fried’s FTX debacle is roiling A.I. research NY Times article - FTX probes $6.5M in payments to AI safety group amid clawback crusade Cointelegraph article - FTX probes $6.5M in payments to AI safety group amid clawback crusade FTX Future Fund article - Future Fund June 2022 Update (archive) Tracxn page - Anthropic Funding and Investors Training & Support Programs Catalyze Impact Seldon Lab SPAR BlueDot Impact YCombinator Pivotal Athena Astra Fellowship Horizon Fellowship BASE Fellowship LASR Labs Entrepeneur First Funding Organizations Coefficient Giving (previously Open Philanthropy) LTFF Longview Philanthropy Renaissance Philanthropy Coworking Spaces LISA Mox Lighthaven FAR Labs Constellation Collider NET Office BAISH Research Organizations & Startups Atla AI Apollo Research Timaeus RAND CAST CHAI Other Sources AXRP website - The AI X-risk Research Podcast LessWrong blogpost - Shard Theory: An Overview
More from Into AI Safety
Etienne Brisson and Allan Brooks, founder and program director of The Human Line Project, sat down with me to talk about AI-induced psychological harm, starting with their own experiences. Allan spent a month spiraling with ChatGPT, ending with him warning national security agencies about an encryption break that never happened. Etienne's uncle became convinced his chatbot was sentient and was hospitalized still messaging it from the back of a police car.
Dr. Evan Miyazono joins Jacob to talk about his path from quantum networking and Protocol Labs to founding Atlas Computing. They discuss why Atlas moved away from formal methods research and toward what Evan calls “field strategy”: identifying important problems, figuring out what interventions could actually address them, and finding the people best positioned to make them happen. The conversation also covers the resilience gap map, coordination failures, AI safety, "value alignment," and the question of how to decide what work is worth doing when there’s no obvious organization responsible for it. Check out the video version of this podcast on the Kairos.fm YouTube channel, or the extended version on Patreon! Chapters
What if the safest AI models weren't built by adding guardrails after training, but by shaping what gets learned in the first place? Ethan Roland, senior alignment researcher at AE Studio and first author on an ICML 2026 spotlight paper, joins Jacob to talk about gradient routing, a technique that routes dangerous capabilities into isolated parts of a model's architecture where they can be locked or removed entirely. They get into the absorption effect, KYC-style access control frameworks, and what it would actually take for frontier labs to adopt this kind of work before it's needed rather than after. Chapters
Today's episode does double duty as an interview and an announcement. Joe Williams, host of the new Kairos.fm show "Beyond the Paycheck: Reclaiming the Case for UBI in the Age of AI," joins Jacob to talk about his background as a freelance translator and how AI quietly dismantled his livelihood in 2025. From there the conversation expands into whether this moment is really different from past waves of automation, who exactly makes up the tech billionaire class, and why you should probably raise an eyebrow when someone like Elon Musk says he supports UBI. Go check out Reclaiming UBI! Chapters
Zainab Majid, co-founder of Asymmetric Security, joins Jacob for a conversation on the intersection between AI Safety and cybersecurity, as well as the future of digital forensics. Drawing from years of incident response work, she explains how cyber attacks actually unfold, why AI is changing both offense and defense, and how her team is building AI-native tools to investigate breaches faster and more effectively. Other topics explored in this episode include trust in the AI/cybersecurity industries, the realities behind cybersecurity hype, and the challenge of keeping humans meaningfully involved as these systems become more capable. Zainab also gives practical, actionable advice on how you can protect yourself. If you're interested in over 30 minutes of additional content, head on over to the Kairos.fm Patreon where you can become a subscriber for just $2 per month, which helps make this whole podcasting thing a bit more sustainable. Chapters