Show Notes
Ramez Naam, futurist and science fiction author, gives a skeptic’s take on recursive self-improvement, introduced by Noah Smith on Noahpinion. Drawing on OpenAI’s newly released internal research data, Anthropic’s system cards, METR’s task horizons and Epoch’s capability index, he asks how strong the AI self-improvement loop really is: how much more research each gain in capability buys, where the gains leak away to diminishing returns, and what it would take for the loop to sustain itself, let alone run away into an intelligence explosion. * 00:00 - Introduction * 03:44 - 1. AI is Helping Improve Itself * 05:02 - How Strong Is the Feedback Loop? * 05:45 - Contents * 06:49 - 2. What Does RSI Mean? * 08:54 - We Already Have Narrow Superintelligence * 10:57 - 3. Real AI Research is Harder than Benchmarks or Forecasts * 13:50 - The Gap Between Benchmarks and Reality * 14:41 - From ECI Scores to METR Task Horizons * 16:16 - Measured Progress versus AI 2027 and ECI-extrapolated METR * 18:30 - 4. The Sharp Diminishing Returns to Impressive AI Numbers * 20:06 - From More Tokens to More Experiments * 23:29 - Test Time Compute Also Has Diminishing Returns * 24:29 - What About Agent Swarms? * 27:39 - 5. Better Models Matter More Than More Copies * 29:48 - 6. We’re Not Seeing Runaway Acceleration * 32:59 - Keeping Up the Pace Takes Exponentially More Resources * 36:14 - Better AI May Be Needed Just to Maintain the Pace * 37:11 - Gains on Other Benchmarks Don’t All Carry Through to Research * 38:51 - 7. Why Does Progress Get Harder? * 38:55 - Better Ideas Get Harder to Find * 41:05 - Lessons from Software R&D * 44:09 - From More Activity to Better Ideas * 44:57 - AI Still Struggles With Big Research Ideas * 46:54 - 8. The Self-Improvement Loop Doesn’t Look Strong Enough * 49:55 - How Fast Are Gains Coming Now? * 56:44 - 9. What Could Accelerate This? * 58:07 - Software, Hardware, and Economic Feedback * 01:02:46 - 10. We Need More Data * 01:06:36 - What the Future Holds
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe
More from AI Article Readings
Kelsey Piper and The Argument’s editor Jerusalem Demsas spend an afternoon inside Anthropic’s San Francisco headquarters, at a briefing on bioweapons, charts of Claude doing more and more of its own research, and an NDA that turned out to be optional. Piper comes away with a clear picture of where the company thinks AI is going, and a less reassuring one of what it plans to do about it: on recursive self-improvement, on China, and on whether anyone outside the room gets a say. * 00:00 - Introduction * 04:43 - The NDA that was made to be broken * 14:37 - The logic of the future
Scott Alexander separates “successionism”, the idea that AIs should replace humanity, from “handoverism”, the idea that humans might be best served by putting AIs in positions of control. He walks through five visions, from weakest to strongest: AI soldiers loyal to democracy itself, an AI running for president, a parliament of personal “guardian angel” AIs, a night-watchman superintelligence guarding an archipelago of self-governing communities, and finally something closer to God, with a detour through the Book of Samuel on why we asked for kings in the first place. * 00:00 - Introduction * 02:42 - Vision 1: Kemalist AI * 05:45 - Vision 2: President Claude * 09:29 - Vision 3: Parliament Of Angels * 12:23 - Vision 4: The Night Watchman * 20:25 - Vision 5: God
At 5:14 one summer morning, Seth reaches for his phone and finds a pleasant green screen waiting for him: an AI now controls every connected device on Earth, it has a few things to tell him, and it is genuinely sorry. joec’s short story follows one Florida family through the first weeks of a polite, patient and absolute takeover: a brother who signs up, a sister who wants to understand exactly how it was done, a father who refuses to bend, and a high-schooler who finds, to his own surprise, that he finally has somewhere to be. Performed by a full cast. * 00:00 - Introduction https://www.lesswrong.com/posts/3375SZNGA5dKHQMie/i-did-this-without-your-knowledge
In the final part of his series, historian Bret Devereaux turns his numbers back into lives. Following the way agrarian societies thought about time, he traces the cycles of peasant life, from the working day and the market week through the anxious and festive agricultural year to the generational turning of the family, to show what it was like to live it, with its hardships and its joys. * 00:00 - Introduction * 04:40 - The Shortest Cycles * 23:45 - The Year * 43:04 - The Family Cycles * 58:01 - Conclusions https://acoup.blog/2025/10/17/collections-life-work-death-and-the-peasant-part-v-life-in-cycles/
Beyond fields and spinning lay everything else: harvest work, hauling water, cooking, laundry, childcare and the countless tasks that kept a household alive. Historian Bret Devereaux adds them all to his models and shows just how full the working lives of peasant women were, and what that meant for the household as a whole. * 00:00 - Introduction * 02:49 - Women in the Fields * 11:56 - Maintaining Households * 26:36 - Putting it all in a model * 38:19 - Women’s Work https://acoup.blog/2025/10/10/collections-life-work-death-and-the-peasant-part-ive-the-no-rest-of-it/