The Generalist

Existential Risk and the Future of Humanity: Lessons from AI, Pandemics, and Nuclear Threats (Toby Ord, Author of "The Precipice")

The Generalist

Show Notes

How close are we to the end of humanity? Toby Ord, Senior Researcher at Oxford University’s AI Governance Initiative and author of The Precipice, argues that the odds of a civilization-ending catastrophe this century are roughly one in six. In this wide-ranging conversation, we unpack the risks that could end humanity’s story and explore why protecting future generations may be our greatest moral duty.

We explore: • Why existential risk matters and what we owe the 10,000-plus generations who came before us • Why Toby believes we face a one-in-six chance of civilizational collapse this century • The four key types of AI risk: alignment failures, gradual disempowerment, AI-fueled coups, and AI-enabled weapons of mass destruction • Why racing dynamics between companies and nations amplify those risks, and how an AI treaty might help • How short-term incentives in democracies blind us to century-scale dangers, along with policy ideas to fix it • The lessons COVID should have taught us (but didn’t) • The hidden ways the nuclear threat has intensified as treaties lapse and geopolitical tensions rise • Concrete steps each of us can take today to steer humanity away from the brink — Transcript: https://www.generalist.com/p/existential-risk-and-the-future-of-humanity-toby-ord — This episode is brought to you by Brex: The banking solution for startups. — Timestamps (00:00) Intro (02:20) An explanation of existential risk, and the study of it (06:20) How Toby’s interest in global poverty sparked his founding of Giving What We Can (11:18) Why Toby chose to study under Derek Parfit at Oxford (14:40) Population ethics, and how Parfit’s philosophy looked ahead to future generations (19:05) An introduction to existential risk (22:40) Why we should care about the continued existence of humans (28:53) How fatherhood sparked Toby’s gratitude to his parents and previous generations (31:57) An explanation of how LLMs and agents work (40:10) The four types of AI risks (46:58) How humans justify bad choices: lessons from the Manhattan Project (51:29) A breakdown of the “unilateralist’s curse” and a case for an AI treaty (1:02:15) Covid’s impact on our understanding of pandemic risk (1:08:51) The shortcomings of our democracies and ways to combat our short-term focus (1:14:50) Final meditations — Follow Toby Ord Website: https://www.tobyord.com/ LinkedIn: https://www.linkedin.com/in/tobyord X: https://x.com/tobyordoxford?lang=en Giving What We Can: https://www.givingwhatwecan.org/ — Resources and episode mentions —Books— • The Precipice: Existential Risk and the Future of Humanity: https://www.amazon.com/dp/0316484911 • Reasons and Persons: https://www.amazon.com/Reasons-Persons-Derek-Parfit/dp/019824908X • Practical Ethics: https://www.amazon.com/Practical-Ethics-Peter-Singer/dp/052143971X

—People— • Derek Parfit: https://en.wikipedia.org/wiki/Derek_Parfit • Carl Sagan: https://en.wikipedia.org/wiki/Carl_Sagan • Stuart Russell: https://en.wikipedia.org/wiki/Stuart_J._Russell

—Other resources— • DeepMind: https://deepmind.google/ • OpenAI: https://openai.com/ • Manhattan Project: https://en.wikipedia.org/wiki/Manhattan_Project • The Unilateralist’s Curse and the Case for a Principle of Conformity: https://nickbostrom.com/papers/unilateralist.pdf • The Nuclear Non-Proliferation Treaty (NPT), 1968: https://history.state.gov/milestones/1961-1968/npt • The Blitz: https://en.wikipedia.org/wiki/The_Blitz • Operation Warp Speed: https://en.wikipedia.org/wiki/Operation_Warp_Speed — Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].

More from The Generalist

Oct 20, 2026

Pete Florence and I began by agreeing that his company has an excellent name. Generalist builds AI for robots, including models that can begin learning a new task from a single demonstration. We talked about dexterity, the strange business of gathering physical experience for a machine, and what we might build if robots could do much more of the work. — Every timestamp 00:00 Intro 02:24 GEN-1.5 and one-shot learning 08:55 What makes a task difficult 10:53 Dexterity in robotics 14:53 Matt Mason’s definition of a hand 17:34 Giving robots a sense of touch 20:30 Growing up in a family of technologists and scientists 22:46 What drew Pete into robotics 27:38 Generalist’s founding thesis and what sets it apart 32:17 Protecting simplicity 35:01 How Generalist’s data collection has evolved 35:30 Generalist’s scaling philosophy 39:53 Generalist’s build-versus-buy philosophy 42:15 Balancing general intelligence with commercial deployment 44:24 Why Pete is betting on dexterity 47:33 The durability of the robotics application layer 49:09 Open-weight robotics models and data flywheels 50:52 Geopolitics, safety, and alignment 52:42 The parallels between teaching humans and robots 1:00:05 The timeline for robotics progress and adoption 1:03:37 Book recommendations — Transcript: https://www.generalist.com/i/220756708/transcript — Pete’s complete reading list The Score Takes Care of Itself: My Philosophy of Leadership: https://www.amazon.com/Score-Takes-Care-Itself-Philosophy/dp/1591843472 Wooden on Leadership: How to Create a Winning Organization: https://www.amazon.com/Wooden-Leadership-Create-Winning-Organization/dp/0071453393 — Other resources Generalist: https://generalistai.com GEN-1.5: https://generalistai.com/blog/gen-1.5 Proprioception: https://en.wikipedia.org/wiki/Proprioception Cutaneous receptor: https://en.wikipedia.org/wiki/Cutaneous_receptor Fukushima nuclear accident: https://en.wikipedia.org/wiki/Fukushima_nuclear_accident The bitter lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf GEN-1: https://generalistai.com/blog/gen-1 — Where to find Pete Florence LinkedIn: https://www.linkedin.com/in/peteflorence X: https://x.com/peteflorence Website: https://www.peteflorence.com — Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].

Oct 8, 2026

Jaan Tallinn is a founding engineer of Skype and Kazaa, and a co-founder of the Cambridge Centre for the Study of Existential Risk and Future of Life Institute. For more than fifteen years, he has been among the most consistent voices arguing that advanced AI poses an extinction-level risk to humanity.

Sep 29, 2026

Liam Fedus and Dogus Cubuk are the co-founders and co-CEOs of Periodic Labs, a startup building “synthesis superintelligence.” In practice, that means an AI system that closes the loop between hypothesis, experiment, and learning.

Sep 8, 2026

Scott Morton is the founder and CEO of Revel, a software platform for testing and controlling complex hardware systems. Before founding the company in 2024, Scott spent nearly a decade at SpaceX building control software for Falcon 9 and Starship. That experience helped him recognize a broader problem: while rockets, nuclear systems, supersonic aircraft, and other advanced machines have grown increasingly sophisticated, the software used to test and control them is often fragmented and decades old. Revel was created to close that gap. The company has raised $180 million, was reportedly valued at just over $1 billion, and counts Impulse Space and Radiant Nuclear among its customers. Its platform scales from small benchtop tests to industrial systems with hundreds of thousands of telemetry channels. Revel has also created its own programming language, RevelCode, designed to combine performance and accessibility with the runtime safety required for high-stakes physical systems.

Aug 18, 2026

Farooq Malik is the co-founder and CEO of Rain, a financial infrastructure company powered by stablecoins. Growing up in an immigrant family, Farooq saw firsthand the friction involved in moving money across borders. Alongside co-founder Charles Yoo-Naut, Farooq spent years building infrastructure before stablecoins became mainstream, betting that tokenized money would eventually become a foundational layer of the global financial system. Today, Rain powers card issuance, payments, and other financial products built on stablecoin rails, helping companies move money faster and operate across markets.

See all episodes →