Show Notes
In this episode of the Existential Hope Podcast, Beatrice Erkers is joined by David Duvenaud, Associate Professor at the University of Toronto and former researcher at Anthropic.
We discuss David’s work on post-AGI civilizational equilibria and the widely discussed paper Gradual Disempowerment. David reflects on why liberalism may not hold up in a world where humans are no longer needed, how UBI could be Goodharted into absurdity, and what it would take to design institutions that protect humans even when incentives don’t.
We also cover:
- Forecasting the long-term future using LLMs trained on historical data - Robin Hanson’s idea of futarchy (governance by prediction markets) - Asymmetrical but beneficial relationships between humans and AI - Uploading, cultural legacies, and the possibility of “worthy successors” On the Existential Hope Podcast hosts Allison Duettmann and Beatrice Erkers from the Foresight Institute invite scientists, founders, and philosophers for in-depth conversations on positive, high-tech futures.
Full transcript, listed resources, and more: https://www.existentialhope.com/podcasts
Follow on X. Hosted on Acast. See acast.com/privacy for more information.
More from The Existential Hope Podcast
We see elections as the foundation of our democracy, but Aristotle thought they create an oligarchy. In ancient Athens, democracy worked by choosing leaders among ordinary citizens by lottery. How could we revive this idea to improve today’s democratic systems?
For 50,000 of generations, humans have passed civilization down to their children. We could be the first generation to hand its future to AIs instead. Should we? Or could the best AI be the kind that chooses to step back and leave some things to us?
Google DeepMind has a 7-hour-long roleplaying game that helps real scientists and policy makers understand how AI will transform science by 2030. And those who played it have found it more informative than any policy brief.
In 2024, Taiwan was flooded with deepfake scam ads. Instead of a crackdown, the government texted 200,000 random citizens: what should we do? People’s proposals became law within months, and by 2025 deepfake ads were down 94%.
There are AIs running labs with little human intervention right now. And they’re doing experiments that human experts would never try. Is this changing what and how science gets done?