Show Notes
Curtis, also known on the internet as AI_WAIFU, is the head of Alignment at EleutherAI. In this episode we discuss the massive orders of H100s from different actors, why he thinks AGI is 4-5 years away, why he thinks we're 90% "toast", his comment on Eliezer Yudkwosky's Death with Dignity, and what kind of Alignment projects is currently going on at EleutherAI, especially a project with Markov chains and the Alignment test project that he is currently leading.
Youtube: https://www.youtube.com/watch?v=9s3XctQOgew
Transcript: https://theinsideview.ai/curtis
Death with Dignity: https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy Alignment Minetest: https://www.eleuther.ai/projects/alignment-minetest Alignment Minetest update: https://blog.eleuther.ai/minetester-intro/
OUTLINE
(00:00) Highlights / Intro
(00:50) The Fuck That Noise Comment On Death With Dignity
(10:28) Th Probability of Doom Is 90%
(12:44) Best Counterarguments For His High P(doom)
(14:41) Compute And Model Size Required For A Dangerous Model
(17:59) Details For Curtis' Model Of Compute Required
(21:23) Why This Estimate Of Compute Required Might Be Wrong, Ajeya Cotra's Transformative AI report
(29:00) Curtis' Median For AGI Is Around 2028, Used To Be 2027
(30:50) How Curtis Approaches Life With Short Timelines And High P(Doom)
(35:27) Takeoff Speeds—The Software view vs. The Hardware View
(39:57) Nvidia's 400k H100 rolling down the assembly line, AIs soon to be unleashed on their own source code
(41:04) Could We Get A Fast Takeoff By Fuly Automating AI Research With More Compute
(46:00) The Entire World (Tech Companies, Governments, Militaries) Is Noticing New AI Capabilities That They Don't Have
(47:57) Open-source vs. Close source policies. Mundane vs. Apocalyptic considerations.
(53:25) Curtis' background, from teaching himself deep learning to EleutherAI
(55:51) Alignment Project At EleutherAI: Markov Chain and Language Models
(01:02:15) Research Philosophy at EleutherAI: Pursuing Useful Projects, Multingual, Discord, Logistics
(01:07:38) Alignment MineTest: Links To Alignmnet, Embedded Agency, Wireheading
(01:15:30) Next steps for Alignment Mine Test: focusing on model-based RL
(01:17:07) Training On Human Data & Using an Updated Gym Environment With Human APIs
(01:19:20) Model Used, Not Observing Symmetry
(01:21:58) Another goal of Alignment Mine Test: Study Corrigibility
(01:28:26) People ordering H100s Are Aware Of Other People Making These Orders, Race Dynamics, Last Message
More from The Inside View
Owain Evans is an AI Alignment researcher, research associate at the Center of Human Compatible AI at UC Berkeley, and now leading a new AI safety research group.
This is a special crosspost episode where Adam Gleave is interviewed by Nathan Labenz from the Cognitive Revolution. At the end I also have a discussion with Nathan Labenz about his takes on AI.
Ethan Perez is a Research Scientist at Anthropic, where he leads a team working on developing model organisms of misalignment.
Emil is the co-founder of palette.fm (colorizing B&W pictures with generative AI) and was previously working in deep learning for Google Arts & Culture.
Evan Hubinger leads the Alignment stress-testing at Anthropic and recently published "Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training".