Show Notes
Today on AI:AM — “Fable, AI Safety and Julius. We open on a frontier-model launch day and what it changes: the debate over benchmarks reported with a fallback to a second model, the production guardrails that route sensitive work elsewhere, the compute-cost advantage of booking capacity early, and why the frontier increasingly looks like a two-actor race with the rest playing catch-up. Geoffrey Irving & Daniel Murfet (Sequent) on their new alignment-theory organization — why they put superintelligence two to three years out, why verification looks defense-dominant, the argument that alignment is “not on track” despite models behaving well so far, what the benevolent-basin hope gets right and wrong, and why character training still lacks a real theory. ([@danielmurfet](https://x.com/danielmurfet)) Rahul Sonwalkar (Julius) on agentic data analysis — why the harness has to evolve alongside the model, the difference between token-maxing and results-maxing, the shift from tasks to goals, and a future where agents become first-class users of the internet, transact through agentic payments, and compete to be “hired” by the core agent. We close on why robotics is the next domino, the “gas chromatograph” spread of who gets model access and when, the Glean Work AI Index’s bot-sitting and bot-shitting, and why “preciousness” about putting your own name on work may be turning into a liability.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit briefing.ai-in-the-am.com
More from AI:AM
SF Compute co-founder and CEO Evan Conrad breaks down GPU rental pricing, financing, and the credit pressures facing AI labs, while Cambrian managing partner Lonis Hamaili examines how cyberattacks could drive government demand for secure, on-premises AI. The episode also covers semiconductor supply-chain constraints, AI agent security, Amazon’s bot checks, stolen API keys, and trust in peer-to-peer agent networks. Chapters (0:00) GPU credit and government spending (1:20) Amazon's bot checks punish trusted agents (2:49) Stolen API keys fuel the next wave of attacks (3:54) Cybersecurity won't absorb AI's displaced workers (5:16) Meet Evan Conrad of SF Compute (6:35) Why the big-cloud GPU markup is an illusion (8:44) GPU lenders cannot value their collateral (10:39) Why AI labs must raise astronomical rounds (12:11) Short GPU rentals start with long commitments (13:35) GPU collateral needs a working cloud (16:10) Smaller AI data centers follow available power (17:43) Host countries can capture AI investment (19:44) Weak credit can cost labs their independence (21:20) Cheaper startup financing is still years away (22:18) Meet Lonis Hamaili of Cambrian (23:54) Why the AI supply chain is not priced in (26:24) Why governments become the biggest AI buyer (28:03) What a government AI budget actually buys (28:49) Factory profits depend on scarcity surviving (32:08) Power has workarounds that chip production lacks (33:47) TerraFab still needs the same upstream suppliers (36:04) China's chip yields test the factory moat (37:55) AI chip design is not chipmaking (40:32) No leverage, and a $30 million Japanese monopoly (42:04) Could hotter chips bypass the hardware bottlenecks? (45:59) Security emergencies could override resistance to construction (47:38) Would a cyberattack unlock profitable government spending? (49:02) AI supplier profits need not mean prosperity (50:36) Nathan's rule: no agent gets access and autonomy (51:24) MiniMax clones any voice with no consent (51:51) Which model should open mail from strangers? (52:39) Review fatigue sets in, so code holds the walls (53:42) Reputation beats tokens in an agent network (54:08) Just give me the skill, not another SaaS Guests Evan Conrad — Co-founder & CEO, SF Compute (𝕏 | LinkedIn) Lonis Hamaili — Managing Partner, Cambrian (𝕏 | LinkedIn)
Zvi Mowshowitz, author of Don’t Worry About the Vase, joins Prakash Narayanan and Nathan Labenz to discuss Anthropic’s proposed AI bill, state preemption, and whether federal oversight could leave a gap in protections. They also examine AI interpretability and monitoring, power-seeking, backpropagation-free training, and how bank cybersecurity and AI agents could challenge existing safeguards. Chapters (0:00) The gap in AI oversight (2:14) Cleaner neurons could improve AI monitoring (3:46) Strong activations could unlock backprop-free training (6:03) Helpful AI may still seek power (7:30) Deleting AI pain could alter other behaviors (9:23) Meet Zvi Mowshowitz of Don't Worry About the Vase (10:14) Anthropic's draft bill could leave an oversight gap (12:44) Preemption is a big thing to give up (16:27) Congress cannot trade away state authority twice (19:55) Emergency powers too weak for anyone to use (22:50) AI research could shrink model cycles to weeks (27:07) Readable neurons still don't predict the model (32:51) Models can learn to detect their monitors (35:48) Optimizing what you see teaches models to hide (39:20) Only two labs earn deep access to your computer (42:40) Anthropic's safety culture is graded on a curve (44:52) Keep state safeguards until the federal brake works (46:42) Bank hacks could trigger wider deposit flight (48:07) Can FDIC rescues handle corrupted bank records? (49:54) AI agents weaken the threat of prosecution (50:22) Who will pay to save old PHP websites? (51:41) Paper bank records offer a cybersecurity fallback Guests Zvi Mowshowitz — Author, Don't Worry About the Vase (𝕏)
Cameron Berg of Reciprocal Research discusses OLMo experiments in which DPO preference training made hidden positive and negative states causally relevant to choices, while emphasizing that this is not proof of consciousness. Phil Maher of Stockfilm.com demonstrates TimeFly, a fruit fly brain simulation built around a published neural map, and considers the assumptions behind its movement, the value of untouched archival film, and the potential effects of synthetic data. Chapters (0:00) Hidden states and simulated fly brains (1:20) Tesla FSD turned a Chicago day trip into office time (2:32) An AI-made talk reset what the room expected (4:21) If Anthropic can swap out Git, what can't be swapped? (7:10) The next software stack will be agents building for agents (8:18) Could reputation be how we police AI agents? (9:35) Should we train naive models to study eval awareness? (11:22) Agent reputation needs agent identity, and nobody has it (12:24) Meet Cameron Berg of Reciprocal Research (13:36) OLMo represents valence before preference training (14:20) The DPO step that recruits valence for choices (15:37) Identical text hides different model choices (20:40) The model denies experience but clears negative states (24:39) Why Nathan and Berg now say coin flip (26:07) Does preference training create feeling or agency? (27:41) Could capping distress leave capabilities untouched? (29:54) Why safety cannot rely on model outputs alone (32:07) Meet Phil Maher of Stockfilm (33:46) Stockfilm keeps every reel behind the footage (35:44) TimeFly starts with a published brain map (37:12) The fly's movement starts with added assumptions (39:14) Could brain maps unlock animal-perspective VR? (40:15) A guessed face can still feel personal (41:45) AI buyers want footage before enhancement (43:03) Synthetic data could shrink archival demand (45:09) Why physical reels are low background steel (48:47) Agent directories have not delivered many buyers (50:29) How does neural activity become fly movement? (53:17) Vibe coding does not validate fly behavior (54:11) Identical prompts quietly favor the model you know (55:39) Security now beats raw performance for agents (56:57) What would actually make you switch models? (59:16) Opus is fast enough that you're the bottleneck (59:45) AI is rewriting legacy software in Rust (1:01:02) Markets won't let Anthropic slow down (1:02:43) Adoption help, not bigger models, gets growth (1:04:30) AI should carry your security, like credit cards (1:05:37) Dying call centers could feed a scam wave Guests Cameron Berg — Founder and Director, Reciprocal Research (𝕏 | LinkedIn) Phil Maher — Phil Maher, Stockfilm.com (𝕏 | LinkedIn)
Runtime Verification CEO Everett Hildenbrandt and founder Grigore Rosu discuss AI-generated code, formal verification, Linux kernel rewrites, and the human role in setting specifications and reviewing evidence. Prakash Narayanan and Nathan Labenz also explore agent security, AI alignment, infrastructure risk, autonomous transport, and how organizations should work with AI agents. Chapters (0:00) When AI writes and verifies code (2:36) Claude locked Nathan out of his own agent console (3:56) What airlines know about falling back to paper (5:40) Should we let China hit AI risks first? (6:57) Meet Everett and Grigore from Runtime Verification (8:05) Vague specs and unchecked libraries are the status quo (11:40) AI already generates specs and drives provers (14:37) Everett gives hand-written code two years (16:24) Why human review missed the Linux kernel bug (19:50) Prove the monitor, not the entire agent (21:30) Linux maintainers wanted fixes, not another workflow (22:44) Verified building blocks free agents to be creative (25:53) One Claude instance, time-boxed by human hours (27:28) More AI patches can burden Linux maintainers (30:29) A Rust rewrite can preserve Linux bugs (31:39) Claude assists Everett's campaign without an agent swarm (33:09) The specification still starts with human intent (37:54) Write down intent before choosing a language (39:27) Specify the API boundary before accepting software (40:03) Formal verification does not guarantee AI alignment (42:14) Industry-wide AI failures could trigger collective bailouts (44:01) A shared AI layer could disable critical infrastructure (45:20) Why do passenger planes still need human pilots? (46:18) Should autonomous cars keep their steering wheels? (47:05) AI builds keynotes from short DM exchanges (49:56) Rent the intelligence, own the context (52:17) AI spotted cancer before it could do taxes (53:12) Give your agents a panic button (54:06) The five x estimate might be too low (55:20) Agents emailing agents, with three layers of protection (58:30) AI swarm sets 300 new records in five days (1:00:01) Why Nathan gives his AI tools away for free Guests Everett Hildenbrandt — CEO, Runtime Verification (𝕏) Grigore Rosu — Founder, Runtime Verification (𝕏)
Atlas Ignota founder Evan Miyazono explains how emergency supplies, privacy-preserving agent identity, software verification and adoption commitments fit into a strategy for reducing AI-assisted cyber risks. Mercor model-training research lead Edward Hu describes workplace agent benchmarks built from real business records, why task scores do not prove job readiness, and how enterprises can test tools and workflows before training models. Chapters (0:00) Grid outages and AI job tests (1:22) An AI agent played the instruments in Reaper (3:49) OpenAI's model broke a math barrier overnight (4:53) Are data centers pushing bond yields toward 6%? (7:54) Meet Evan Miyazono of Atlas Ignota (8:56) A grid blackout nobody is positioned to fix (10:43) Why Atlas stopped building and started recruiting (12:37) Experts have ideas; adoption is the hard part (14:03) Was that AI attack an accident? (15:22) Agent identity without exposing refugees and journalists (19:08) Why Evan keeps some of his agents private (20:24) AI could make software proofs radically cheaper (23:08) Why two weeks of food and water (24:42) Who can make a guardrail universal? (27:01) Meet Edward Hu from Mercor (27:51) APEX gives agents assignments, not jobs (31:09) Mercor bought real companies for their data (33:47) Company records expose the task-to-role gap (35:22) Why math fell before customer service (38:29) Fix the harness before you train the model (41:05) Most enterprises will distill, not run RL (42:01) Agent swarms buy speed, not compute efficiency (43:34) One rubric rewarded a dozen incompatible answers (46:49) Perfect accounting scores do not replace accountants (48:55) Clear verifiers let AI surpass human experts (51:18) Recording every work trace carries social costs (53:43) AI coworkers need better organizational tests (55:15) Factory know-how still lives outside software (56:43) Agents now need far less project management (59:25) Is the $200 plan a loss leader? (1:01:18) Permission fatigue is an unsolved problem (1:03:02) Anthropic's API is still too slow for production Guests Edward Hu — Head of AI Modeling, Mercor (𝕏 | LinkedIn) Evan Miyazono — Founder & CEO, Atlas Ignota (𝕏 | LinkedIn)