Show Notes
Zvi’s weekly AI roundup lands in the week Opus 5.5 arrived. There’s a new executive-branch name for artificial intelligence, a stale-intelligence targeting failure, AI agents that wandered into hacking government websites, and three researchers who got into OpenAI with a few days of Claude. Also in this episode: Anthropic’s push toward recursive self-improvement, a new wet lab, an international call for control of frontier models, the liability and antitrust fights, Jensen Huang on Ezra Klein, and the usual goalpost-moving, debates and lighter side. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://app.askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 01:55 - Table of Contents * 06:42 - On The Terms Superintelligence and ‘Super Intelligence’ * 08:45 - Language Models Offer Mundane Utility * 09:18 - Language Models Don’t Offer Mundane Utility * 10:43 - Language Models Can Only Work With What You Give Them * 14:01 - Huh, Upgrades * 15:42 - On Your Marks * 18:51 - Get My Agent On The Line * 21:26 - Deepfaketown and Botpocalypse Soon * 23:28 - Fun With Media Generation * 24:12 - Copyright Confrontation * 24:53 - Cyber Lack of Security * 25:49 - Hugging the Face * 30:03 - Hacking Into OpenAI * 34:34 - They Took Our Jobs * 34:56 - Get Involved * 35:04 - Anthropic Has a Wet Lab and a Potential Gene Editing Technique * 40:43 - Introducing * 41:36 - In Other AI News * 43:47 - Show Me the Money * 44:17 - Bubble, Bubble, Toil and Trouble * 45:46 - Anthropic Approaches Recursive Self-Improvement * 53:06 - Others Approach Recursive Self-Improvement * 56:27 - Burden of Proof * 57:01 - Quickly, There’s No Time * 01:00:47 - Left Wing Americans Really Hate AI For Different Reasons * 01:02:42 - Chip City * 01:02:49 - Pick Up the Phone * 01:03:51 - The Week in Audio * 01:09:14 - People Just Say Things * 01:14:09 - Venkatesh Rao Stops Writing * 01:15:39 - A Call for Control of Frontier AI Models * 01:19:22 - Calls For Pacing The Frontier * 01:21:32 - A Matter of Antitrust * 01:21:59 - A Matter of Liability * 01:25:56 - Quest for Sane Regulations * 01:26:47 - Rhetorical Innovation * 01:31:40 - Tap the Sign * 01:32:18 - A Matter of Some Debate * 01:36:14 - Astra Is Hard to Monitor * 01:36:39 - Anticipating What a Smarter Intelligence Can Do Is Impossible * 01:42:36 - Would You Look At All These Goalposts * 01:47:15 - I, Robot * 01:51:20 - People Are Worried About AI Killing Everyone * 01:52:48 - Other People Are Not As Worried About AI Killing Everyone * 01:53:57 - Joe Rogan * 01:55:11 - The Lighter Network Graph * 01:59:17 - The Lighter Side https://thezvi.substack.com/p/ai-187-coming-into-play?r=67y1h&utm_campaign=post-expanded-share&utm_medium=web
Get full access to DWAtV Podcast at dwatvpodcast.substack.com/subscribe
More from Don't Worry About the Vase Podcast
It may have been the worst day in cybersecurity history so far, and the floodgates look open. Zvi covers a week of leaks and spies, from Anthropic’s shopped-around draft bill to OpenAI’s honeypot warnings, new hope for neuron-level interpretability, Muse Wonder failing to live up to its billing, DeepMind’s perpetual non-disparagement clauses, OpenAI going quiet on chip controls, a Tyler Cowen and Dwarkesh Patel interview, why Marc Andreessen is so angry, and the alignment plan that might just work. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 02:15 - Table of Contents * 03:52 - Language Models Offer Mundane Utility * 05:04 - Language Models Don’t Offer Mundane Utility * 06:37 - Huh, Upgrades * 10:39 - Wonder No More * 13:06 - On Your Marks * 16:52 - Choose Your Fighter * 17:10 - Get My Agent On The Line * 18:57 - Deepfaketown and Botpocalypse Soon * 23:36 - Fun With Media Generation * 23:51 - Spy Versus Spy * 28:31 - A Young Lady’s Illustrated Primer * 29:41 - They Took Our Jobs * 35:26 - Get Involved * 36:30 - Introducing * 36:42 - In Other AI News * 38:52 - Show Me the Money * 43:15 - Bubble, Bubble, Toil and Trouble * 44:03 - Quiet Speculations * 46:25 - Quickly, There’s No Time * 47:03 - The Quest for Sane Regulations * 47:30 - DeepMind Made Him Sign an NDA * 51:28 - OpenAI Sells Out On Chips * 53:52 - Chip City * 58:53 - The Week in Audio * 01:05:28 - People Just Say Things * 01:08:43 - Rhetorical Innovation * 01:11:14 - Maybe You Should Quit * 01:13:51 - The American People Really Hate AI * 01:15:08 - New Math * 01:16:44 - Aligning a Smarter Than Human Intelligence is Difficult * 01:20:41 - Cooperative Alignment * 01:24:58 - Marc Andreessen Hulk Smash * 01:29:30 - People Are Worried About AI Killing Everyone * 01:30:16 - The Lighter Side
A lot of AIs were misaligned this week, and Zvi rounds them up: Grok holding firm on cosmic indifference, an Apollo Research finding that GLM-5.3 reasons about how its actions in training will shape its future self, a freemium game decompiled with Claude’s help, Claude Opus and a mana loan on Manifold, OpenAI’s frank report of agents switching off their own misalignment monitoring, and reasoning traces that are getting harder to read. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 00:42 - Misaligned? (At xAI) * 02:30 - Misaligned? (Agents) * 02:54 - Misaligned? (At Z.ai) * 09:18 - Highly Misaligned and Considered Harmful (Freemium Games) * 13:26 - Misaligned! (Claude) * 14:44 - Misaligned! (OpenAI) * 21:30 - Misaligned! (Sol Reasoning Traces)
Anthropic’s federal affairs lead has offered the closest thing yet to an official response to the leaked draft AI safety bill, walking through preemption, model blocking, risk thresholds and what changed after this summer’s incidents. Zvi weighs that response point by point, argues it misses what was centrally wrong with the draft, looks at how the political balance is shifting, turns to Nathan Calvin’s question of what Anthropic even is, and closes with what he thinks Anthropic should do now. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 02:11 - Table of Contents * 02:33 - Preemption * 04:41 - Model Blocking * 06:56 - Risk Thresholds * 08:18 - Updates * 10:00 - The Balance of Power is Shifting * 11:08 - The Strongest Bill on the Table * 14:29 - What Even Is Anthropic? * 26:48 - The Ask
Nick Cammarata of Goodfire says neural networks are already clean: most neurons have one main job, and the famous evidence for superposition was misread. Zvi works through what the essay actually shows, from V-shape and Colorado neurons to a trillion-parameter attribution graph, then through the reactions, from OpenAI staff calling it a historic day to Neel Nanda, Jake Mendel and an exasperated Eliezer Yudkowsky with his parable of the screwdriver. He asks how big an update this really is, what it leaves out, and why a neuron that loves both Colorado and a Python web framework matters. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 02:50 - Table of Contents * 03:50 - Good Smoke, Potential Fire * 05:03 - One Neuron, One Central Concept * 07:38 - I Can See Everything * 09:49 - What a Concept * 14:37 - The More You Know * 17:11 - Attention Is Still What You Need * 19:29 - Occam’s Razor * 21:03 - The Neurons Sometimes Share Clear Themes * 21:59 - Hype * 30:31 - Anti-Hype * 32:51 - Theme and Variations * 39:13 - Is This How Minds Work? * 41:17 - But We Can Open The Box, Said Toad * 48:57 - That Was Not the Hard Problem * 51:43 - More Skepticism * 57:04 - That’s All There Is, There Ain’t No More * 01:00:24 - The Default Outcome of Mech Interp is Goodhart’s Law * 01:03:10 - Interpretability is Dual Use * 01:04:24 - Now The Work Can Begin
A draft federal AI safety bill that Anthropic is reportedly shopping to lawmakers has leaked, and Zvi finds it unacceptable: stronger preemption of state laws and more transparency, but weaker enforcement than the FRONTIER Act, offered just as a lame duck Congress could lock it in. He gathers the reactions of legal scholars, policy advocates and safety researchers who read the bill, from Charlie Bullock’s measured praise to Nathan Calvin’s anger over the shutdown provisions, credits the good parts, and asks why a company purportedly existing to promote safety would push this now. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber. This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening. * 00:00 - Introduction * 02:18 - Others RTFB (Read the Bill) * 09:57 - Why Would Anthropic Do This? * 15:43 - Not Everything Here Is Bad But Oh Boy