The Generalist

Jaan Tallinn would like us to survive

The Generalist

Show Notes

Jaan Tallinn is a founding engineer of Skype and Kazaa, and a co-founder of the Cambridge Centre for the Study of Existential Risk and Future of Life Institute. For more than fifteen years, he has been among the most consistent voices arguing that advanced AI poses an extinction-level risk to humanity.

Jaan became interested in AI risk in 2008 after reading the work of Eliezer Yudkowsky, a founding researcher of the field of AI alignment. In 2009, after a four-hour meeting with Yudkowsky, he made his first donation to what became the Machine Intelligence Research Institute (MIRI). He went on to become an early investor in DeepMind and led the first round into Anthropic – despite believing that AI development, if unchecked, could kill us all. His reasoning: his money displaced VC money rather than bringing Anthropic into existence, and the influence and proceeds that come with it help fund safety work.

In this conversation, we explore what Jaan believes the OpenAI/Hugging Face attack revealed about AI motivation and why he was less surprised than most. We discuss his evolving risk estimate, his case for an immediate moratorium on frontier model training, the technical and geopolitical case for hardware-based verification as a foundation for global AI governance, and why he asked Dario Amodei to collect competing term sheets before accepting his own investment. — Transcript: https://www.generalist.com/i/218944380/transcript — Every timestamp (00:00) Intro (02:28) Growing up in Soviet-occupied Estonia (05:54) How Jaan assesses AI risk and why he’s more optimistic now (07:16) AI’s preferences and motivations (12:44) The OpenAI Hugging Face incident (16:12) Meeting with Eliezer Yudkowsky (18:20) Everything as computation (20:22) The Coxon moment (23:05) Who dismisses AI alignment risks (26:17) The AI race, the case for a pause, and safer alternatives (32:47) AI 2027 (33:44) Investing in DeepMind and Anthropic (40:03) The “big blob of compute” and competing approaches to AI (44:42) What effective AI safeguards could look like (48:15) Verifying countries’ compliance with AI safeguards (50:15) AI kill switches and their limits (52:11) Jaan’s love of dance and musical influences (59:50) Jaan’s favorite movies (1:01:41) What Jaan likes about AI — Jaan’s complete reading list Shadows of the Mind: A Search for the Missing Science of Consciousness: https://www.amazon.com/dp/0195106466 — Other resources Skype: https://en.wikipedia.org/wiki/Skype Life is NOT a Journey - Alan Watts: https://www.youtube.com/watch?v=rBpaUICxEhk&t=10s AI Billionaire on Existential Risk: Jaan Tallinn – #112: https://www.manifold1.com/episodes/ai-billionaire-on-existential-risk-jaan-tallinn The Basic AI Drives: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf The Hugging Face incident and the road ahead: https://openai.com/index/hugging-face-incident-and-the-road-ahead Rogue AI didn’t breach Hugging Face, human decisions did: https://thebulletin.org/2026/09/rogue-ai-didnt-breach-hugging-face-human-decisions-did MIRI: https://intelligence.org Jacob Coxon on X, “I resigned from Anthropic today.”: https://x.com/hilbertspaess/status/2097476196791709843 AI 2027: https://ai-2027.com Google DeepMind: https://deepmind.google Inside the Biggest Feud in Artificial Intelligence: https://www.theatlantic.com/technology/2026/09/openai-v-anthropic-inside-biggest-rivalry-tech/688819 The Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html AI 2040: https://ai-2040.com The Glass Bead Game: https://www.amazon.com/dp/B00DO994WY Jaan’s favorite movies: https://jaan.info/favmovies The Odyssey: https://www.imdb.com/title/tt33764258 On the Edge: https://www.imdb.com/title/tt0221559 Limitless: https://www.imdb.com/title/tt1219289 — Where to find Jaan Tallinn Website: https://jaan.info — Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].

More from The Generalist

Oct 20, 2026

Pete Florence and I began by agreeing that his company has an excellent name. Generalist builds AI for robots, including models that can begin learning a new task from a single demonstration. We talked about dexterity, the strange business of gathering physical experience for a machine, and what we might build if robots could do much more of the work. — Every timestamp 00:00 Intro 02:24 GEN-1.5 and one-shot learning 08:55 What makes a task difficult 10:53 Dexterity in robotics 14:53 Matt Mason’s definition of a hand 17:34 Giving robots a sense of touch 20:30 Growing up in a family of technologists and scientists 22:46 What drew Pete into robotics 27:38 Generalist’s founding thesis and what sets it apart 32:17 Protecting simplicity 35:01 How Generalist’s data collection has evolved 35:30 Generalist’s scaling philosophy 39:53 Generalist’s build-versus-buy philosophy 42:15 Balancing general intelligence with commercial deployment 44:24 Why Pete is betting on dexterity 47:33 The durability of the robotics application layer 49:09 Open-weight robotics models and data flywheels 50:52 Geopolitics, safety, and alignment 52:42 The parallels between teaching humans and robots 1:00:05 The timeline for robotics progress and adoption 1:03:37 Book recommendations — Transcript: https://www.generalist.com/i/220756708/transcript — Pete’s complete reading list The Score Takes Care of Itself: My Philosophy of Leadership: https://www.amazon.com/Score-Takes-Care-Itself-Philosophy/dp/1591843472 Wooden on Leadership: How to Create a Winning Organization: https://www.amazon.com/Wooden-Leadership-Create-Winning-Organization/dp/0071453393 — Other resources Generalist: https://generalistai.com GEN-1.5: https://generalistai.com/blog/gen-1.5 Proprioception: https://en.wikipedia.org/wiki/Proprioception Cutaneous receptor: https://en.wikipedia.org/wiki/Cutaneous_receptor Fukushima nuclear accident: https://en.wikipedia.org/wiki/Fukushima_nuclear_accident The bitter lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf GEN-1: https://generalistai.com/blog/gen-1 — Where to find Pete Florence LinkedIn: https://www.linkedin.com/in/peteflorence X: https://x.com/peteflorence Website: https://www.peteflorence.com — Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].

Sep 29, 2026

Liam Fedus and Dogus Cubuk are the co-founders and co-CEOs of Periodic Labs, a startup building “synthesis superintelligence.” In practice, that means an AI system that closes the loop between hypothesis, experiment, and learning.

Sep 8, 2026

Scott Morton is the founder and CEO of Revel, a software platform for testing and controlling complex hardware systems. Before founding the company in 2024, Scott spent nearly a decade at SpaceX building control software for Falcon 9 and Starship. That experience helped him recognize a broader problem: while rockets, nuclear systems, supersonic aircraft, and other advanced machines have grown increasingly sophisticated, the software used to test and control them is often fragmented and decades old. Revel was created to close that gap. The company has raised $180 million, was reportedly valued at just over $1 billion, and counts Impulse Space and Radiant Nuclear among its customers. Its platform scales from small benchtop tests to industrial systems with hundreds of thousands of telemetry channels. Revel has also created its own programming language, RevelCode, designed to combine performance and accessibility with the runtime safety required for high-stakes physical systems.

Aug 18, 2026

Farooq Malik is the co-founder and CEO of Rain, a financial infrastructure company powered by stablecoins. Growing up in an immigrant family, Farooq saw firsthand the friction involved in moving money across borders. Alongside co-founder Charles Yoo-Naut, Farooq spent years building infrastructure before stablecoins became mainstream, betting that tokenized money would eventually become a foundational layer of the global financial system. Today, Rain powers card issuance, payments, and other financial products built on stablecoin rails, helping companies move money faster and operate across markets.

Aug 4, 2026

Eric Nguyen is the co-founder and CEO of Radical Numerics, an AI research lab that has raised $50 million to train models directly on biological data. Before starting the company, Eric helped develop Evo and Evo 2, large-scale genome language models trained on unlabeled DNA sequences. Radical Numerics is now building models that can connect information across DNA, RNA, proteins, epigenetics, and other parts of biology, rather than treating each as a separate problem. Researchers have already used Evo to generate viable bacteriophage genomes, and Eric says Radical Numerics’ newer model, Omnii, matched key findings from two years of Alzheimer’s wet-lab research in a matter of days. He also believes these tools could make it easier to create dangerous pathogens, which is why the company is working on both biological design and biodefense.

See all episodes →