Show Notes
Rose Hadshar (Forethought) and Nora Ammann (ARIA) talk about power concentration, checks and balances, coups, and post-AGI political economy.
You can read a full transcript here.
This is the audio version of a video podcast. Watch the video on YouTube: youtu.be/PB855Xpx1kk?si=v05LVM-2VQ3XiT7N
Prompted in part by Rose's recent article for 80,000 Hours on ‘Extreme power concentration’ — 80000hours.org/problem-profiles/extreme-power-concentration
Timstamps:
00:00:00 Intro
00:01:26 What is ‘power concentration’? Is power concentration the right framing?
00:05:25 When can it be good for power to be centralised?
00:09:40 Is ‘checks and balances’ a better framing?
00:11:16 How AI undermines existing checks and balances
00:15:25 Economic power, meme complexes, and cultural influence
00:28:50 AI companies vs governments as centres of power
00:31:25 The difficulty of knowing where power actually lies
00:40:05 Do humans and AIs concentrate power differently?
00:48:38 Should we be trying to imagine a better post-AGI political economy?
00:56:28 Concrete actions: transparency & whistleblower protections
00:59:07 Human-AI teaming and collaborative intelligence
01:01:16 AI agent economies and proving trustworthiness
01:13:16 Can we train AI to follow the law?
01:16:33 Building resilient coalitions of trustworthy agents
01:18:42 Closing reflections
More from ForeCast
Will MacAskill (Senior Research Fellow at Forethought) and Mia Taylor (Research Fellow at Forethought) discuss how to use philanthropic money to do the most good during and after the intelligence explosion, and explain why wealthy donors should consider handing off their philanthropy. They talk about:
Matthew Adelstein was a Visiting Scholar at Forethought and blogs as Bentham’s Bulldog, where he writes on ethics, animal welfare, and the long-run future.
Peter Salib is an Assistant Professor of Law at the University of Houston Law Center and co-director of the Center for Law & AI Risk. Simon Goldstein is an associate professor at the University of Hong Kong, a senior editor at AI Frontiers, and a visiting senior scholar at Forethought. They have written together about AI safety, AI rights, and the governance of advanced AI.
Will MacAskill is a senior research fellow at Forethought and the author of What We Owe The Future. Tom Davidson is a senior research fellow at Forethought and the author of a series of reports on AI timelines, takeoff speeds, and AI-enabled coups.
Avi Parrack is a physics PhD student at Stanford and a researcher at Forethought, where he works on AI, space expansion, and governance. He is the lead author of a recent Forethought report on orbital data centers.