Show Notes
As AI systems become more capable and autonomous, staying in control of what they do is increasingly difficult. But the AI we need to solve problems like cancer doesn't have to be one we can no longer steer. In this episode, we talk with Anthony Aguirre, cosmologist and co-founder of the Future of Life Institute, about how we can design AI to amplify human capability rather than substitute for it. We cover: The economic driver Anthony sees behind AGI: largely not scientific breakthroughs, but capturing a share of the global labor market What "Tool AI" means to Anthony as an alternative to AGI, and why he thinks it can deliver most of what we want without replacing people Whether Tool AI is stable: the tension between staying in control of AI and the ease of completing tasks How legal liability for AI agents could quietly steer the industry toward more controllable systems Two concrete ideas for transformative AI tools we could build today to improve democracy and the information landscape This episode is part of our AI Pathways series, where we explore the choices we can still make about how AI gets built.
Chapters: 0:00 Cold open 0:43 Why we should build detailed scenarios of alternative AI futures 4:42 Turning abstract AI philosophies into concrete futures: the value of scenario building 6:42 AI as a tool vs. AI as a replacement: the case for keeping humans in control 13:27 Is Tool AI a stable long-term path, or will it drift toward AGI? 16:03 Why the AI industry is heading towards replacing humans, and how to shift that 23:41 What should AI replace, and what should stay human? 29:20 How AI liability law could force a shift toward safer, controllable systems 35:16 Tool AI vs. comprehensive AI services: two visions for a safer AI future 39:50 Two AI tools we could build today: large-scale deliberation and the epistemic stack
On the Existential Hope Podcast hosts Allison Duettmann and Beatrice Erkers from the Foresight Institute invite scientists, founders, and philosophers for in-depth conversations on positive, high-tech futures.
Full transcript, listed resources, and more: https://www.existentialhope.com/podcasts
Follow on X. Hosted on Acast. See acast.com/privacy for more information.
More from The Existential Hope Podcast
We see elections as the foundation of our democracy, but Aristotle thought they create an oligarchy. In ancient Athens, democracy worked by choosing leaders among ordinary citizens by lottery. How could we revive this idea to improve today’s democratic systems?
For 50,000 of generations, humans have passed civilization down to their children. We could be the first generation to hand its future to AIs instead. Should we? Or could the best AI be the kind that chooses to step back and leave some things to us?
Google DeepMind has a 7-hour-long roleplaying game that helps real scientists and policy makers understand how AI will transform science by 2030. And those who played it have found it more informative than any policy brief.
In 2024, Taiwan was flooded with deepfake scam ads. Instead of a crackdown, the government texted 200,000 random citizens: what should we do? People’s proposals became law within months, and by 2025 deepfake ads were down 94%.
There are AIs running labs with little human intervention right now. And they’re doing experiments that human experts would never try. Is this changing what and how science gets done?