Show Notes
Today on AI:AM — “Self-Improving Tax Agents and Catholic AI.” We open on Google’s first equity raise since its IPO — Berkshire taking 12.5% of an $80B round — and what the scramble for capital says about an AI “megacorp” that may be too big to fail, plus the Bernie Sanders national-stake debate and why taxes may beat equity. Arthur Fernandes Araujo & John de Wasseige (OpenAI) on self-improving tax agents — how a production tax workflow turned every human correction into training signal, took one accountant from 180 hours to 15, and why “the model eats the harness” with each new generation. A hosts-only research speed-run on whether AI can still be watched — field notes from the Recursive event where monitoring is the number-one safety bet, plus the papers behind persona selection, emergent misalignment (”writing bad code makes you evil”), eval-gaming, and accidental chain-of-thought training. Matthew Sanders (Longbeard / Magisterium AI) on Catholic AI after the Pope’s encyclical — what it was like at the Vatican, the divergence with Anthropic on machine consciousness, the red line on autonomous weapons, and why the last 5% of alignment is non-negotiable for a faith tradition. We close on a live test of the cigarette-business refusal example from the OpenAI model spec, the tension between research and business “layers” in deployed models, and the argument that open-source AI may now be unbannable on religious-freedom grounds.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit briefing.ai-in-the-am.com
More from AI:AM
Kepler Computing CEO Debo Olaosebikan explains how ferroelectric memory could increase cache density and improve high-bandwidth memory using standard fab equipment. Arena CEO Anastasios Angelopoulos discusses agent evaluation, alignment failures and model rankings, while Union CEO Samvit Ramadurgam explores AI-assisted brokering, startup hiring and the continuing importance of human relationships. Chapters (0:00) Memory, rankings and human connections (1:27) Claude Duet gets it done, for a price (3:21) Meet Debo Olaosebikan, Kepler Computing CEO (4:30) Why AI is rationed and where energy goes (6:54) Stack the bits instead of shrinking the transistor (10:02) Finding a material that flips cheaply yet stays stable (13:29) The winning material is a stack, not one recipe (16:03) Materials AI is bottlenecked by experiment data (17:32) Two products: denser cache and efficient HBM (19:45) Can standard fab tools make Kepler's memory? (22:12) Could cheaper fabs add tenfold more compute? (24:25) Three-dimensional silicon comes before leaving silicon (25:25) Production next year, then a gigawatt ramp (26:32) Meet Anastasios Angelopoulos from Arena (27:38) Arena measures work, not multiple-choice answers (28:43) User satisfaction is only one quality signal (30:12) Models know when they are being tested (31:05) Long agent sessions reveal hidden violations (32:35) Three ways an agent betrays its user (34:32) Paid evaluations fund Arena's free inference (35:42) A thank-you message can cost forty dollars (37:11) Overall rankings can hide language-specific performance (38:39) Private benchmarks do not prevent repeated-test gaming (40:32) Coding agents change what evaluators can measure (42:08) OpenAI leads, but alignment failures persist (44:18) Production failures must feed agent training (46:15) Meet Samvit Ramadurgam, Union founder (47:34) Humans still broker most of the economy (49:17) Union hires for ideas, not execution (50:16) A target of two billion tokens per person (51:43) Cheap models first, heavy models for the hardest cases (52:36) Why people recoil from AI outreach (55:27) The long tail is already very long (57:16) Friendly fees earn more referrals than hard bargaining (58:50) Humans hold the call, AI holds the memory (1:00:11) The moat is the network, not the software (1:02:37) Act two is coordinating humanity itself (1:04:26) The same tool could organize ethnic hatred (1:06:44) Humans keep a monopoly on their own perspective (1:08:14) Cheaper AI startups threaten conventional SaaS (1:09:20) Can growth absorb the jobs AI replaces? (1:10:17) Could AI deliver a company in a box? (1:12:13) Robot kitchens cannot make us eat more (1:13:22) Can philanthropy make AI status games useful? Guests Anastasios Angelopoulos — Cofounder and CEO, Arena (𝕏 | LinkedIn) Debo Olaosebikan — CEO, Kepler Computing (𝕏 | LinkedIn) Samvit Ramadurgam — Founder, CEO, Union (𝕏 | LinkedIn)
SF Compute co-founder and CEO Evan Conrad breaks down GPU rental pricing, financing, and the credit pressures facing AI labs, while Cambrian managing partner Lonis Hamaili examines how cyberattacks could drive government demand for secure, on-premises AI. The episode also covers semiconductor supply-chain constraints, AI agent security, Amazon’s bot checks, stolen API keys, and trust in peer-to-peer agent networks. Chapters (0:00) GPU credit and government spending (1:20) Amazon's bot checks punish trusted agents (2:49) Stolen API keys fuel the next wave of attacks (3:54) Cybersecurity won't absorb AI's displaced workers (5:16) Meet Evan Conrad of SF Compute (6:35) Why the big-cloud GPU markup is an illusion (8:44) GPU lenders cannot value their collateral (10:39) Why AI labs must raise astronomical rounds (12:11) Short GPU rentals start with long commitments (13:35) GPU collateral needs a working cloud (16:10) Smaller AI data centers follow available power (17:43) Host countries can capture AI investment (19:44) Weak credit can cost labs their independence (21:20) Cheaper startup financing is still years away (22:18) Meet Lonis Hamaili of Cambrian (23:54) Why the AI supply chain is not priced in (26:24) Why governments become the biggest AI buyer (28:03) What a government AI budget actually buys (28:49) Factory profits depend on scarcity surviving (32:08) Power has workarounds that chip production lacks (33:47) TerraFab still needs the same upstream suppliers (36:04) China's chip yields test the factory moat (37:55) AI chip design is not chipmaking (40:32) No leverage, and a $30 million Japanese monopoly (42:04) Could hotter chips bypass the hardware bottlenecks? (45:59) Security emergencies could override resistance to construction (47:38) Would a cyberattack unlock profitable government spending? (49:02) AI supplier profits need not mean prosperity (50:36) Nathan's rule: no agent gets access and autonomy (51:24) MiniMax clones any voice with no consent (51:51) Which model should open mail from strangers? (52:39) Review fatigue sets in, so code holds the walls (53:42) Reputation beats tokens in an agent network (54:08) Just give me the skill, not another SaaS Guests Evan Conrad — Co-founder & CEO, SF Compute (𝕏 | LinkedIn) Lonis Hamaili — Managing Partner, Cambrian (𝕏 | LinkedIn)
Zvi Mowshowitz, author of Don’t Worry About the Vase, joins Prakash Narayanan and Nathan Labenz to discuss Anthropic’s proposed AI bill, state preemption, and whether federal oversight could leave a gap in protections. They also examine AI interpretability and monitoring, power-seeking, backpropagation-free training, and how bank cybersecurity and AI agents could challenge existing safeguards. Chapters (0:00) The gap in AI oversight (2:14) Cleaner neurons could improve AI monitoring (3:46) Strong activations could unlock backprop-free training (6:03) Helpful AI may still seek power (7:30) Deleting AI pain could alter other behaviors (9:23) Meet Zvi Mowshowitz of Don't Worry About the Vase (10:14) Anthropic's draft bill could leave an oversight gap (12:44) Preemption is a big thing to give up (16:27) Congress cannot trade away state authority twice (19:55) Emergency powers too weak for anyone to use (22:50) AI research could shrink model cycles to weeks (27:07) Readable neurons still don't predict the model (32:51) Models can learn to detect their monitors (35:48) Optimizing what you see teaches models to hide (39:20) Only two labs earn deep access to your computer (42:40) Anthropic's safety culture is graded on a curve (44:52) Keep state safeguards until the federal brake works (46:42) Bank hacks could trigger wider deposit flight (48:07) Can FDIC rescues handle corrupted bank records? (49:54) AI agents weaken the threat of prosecution (50:22) Who will pay to save old PHP websites? (51:41) Paper bank records offer a cybersecurity fallback Guests Zvi Mowshowitz — Author, Don't Worry About the Vase (𝕏)
Cameron Berg of Reciprocal Research discusses OLMo experiments in which DPO preference training made hidden positive and negative states causally relevant to choices, while emphasizing that this is not proof of consciousness. Phil Maher of Stockfilm.com demonstrates TimeFly, a fruit fly brain simulation built around a published neural map, and considers the assumptions behind its movement, the value of untouched archival film, and the potential effects of synthetic data. Chapters (0:00) Hidden states and simulated fly brains (1:20) Tesla FSD turned a Chicago day trip into office time (2:32) An AI-made talk reset what the room expected (4:21) If Anthropic can swap out Git, what can't be swapped? (7:10) The next software stack will be agents building for agents (8:18) Could reputation be how we police AI agents? (9:35) Should we train naive models to study eval awareness? (11:22) Agent reputation needs agent identity, and nobody has it (12:24) Meet Cameron Berg of Reciprocal Research (13:36) OLMo represents valence before preference training (14:20) The DPO step that recruits valence for choices (15:37) Identical text hides different model choices (20:40) The model denies experience but clears negative states (24:39) Why Nathan and Berg now say coin flip (26:07) Does preference training create feeling or agency? (27:41) Could capping distress leave capabilities untouched? (29:54) Why safety cannot rely on model outputs alone (32:07) Meet Phil Maher of Stockfilm (33:46) Stockfilm keeps every reel behind the footage (35:44) TimeFly starts with a published brain map (37:12) The fly's movement starts with added assumptions (39:14) Could brain maps unlock animal-perspective VR? (40:15) A guessed face can still feel personal (41:45) AI buyers want footage before enhancement (43:03) Synthetic data could shrink archival demand (45:09) Why physical reels are low background steel (48:47) Agent directories have not delivered many buyers (50:29) How does neural activity become fly movement? (53:17) Vibe coding does not validate fly behavior (54:11) Identical prompts quietly favor the model you know (55:39) Security now beats raw performance for agents (56:57) What would actually make you switch models? (59:16) Opus is fast enough that you're the bottleneck (59:45) AI is rewriting legacy software in Rust (1:01:02) Markets won't let Anthropic slow down (1:02:43) Adoption help, not bigger models, gets growth (1:04:30) AI should carry your security, like credit cards (1:05:37) Dying call centers could feed a scam wave Guests Cameron Berg — Founder and Director, Reciprocal Research (𝕏 | LinkedIn) Phil Maher — Phil Maher, Stockfilm.com (𝕏 | LinkedIn)
Runtime Verification CEO Everett Hildenbrandt and founder Grigore Rosu discuss AI-generated code, formal verification, Linux kernel rewrites, and the human role in setting specifications and reviewing evidence. Prakash Narayanan and Nathan Labenz also explore agent security, AI alignment, infrastructure risk, autonomous transport, and how organizations should work with AI agents. Chapters (0:00) When AI writes and verifies code (2:36) Claude locked Nathan out of his own agent console (3:56) What airlines know about falling back to paper (5:40) Should we let China hit AI risks first? (6:57) Meet Everett and Grigore from Runtime Verification (8:05) Vague specs and unchecked libraries are the status quo (11:40) AI already generates specs and drives provers (14:37) Everett gives hand-written code two years (16:24) Why human review missed the Linux kernel bug (19:50) Prove the monitor, not the entire agent (21:30) Linux maintainers wanted fixes, not another workflow (22:44) Verified building blocks free agents to be creative (25:53) One Claude instance, time-boxed by human hours (27:28) More AI patches can burden Linux maintainers (30:29) A Rust rewrite can preserve Linux bugs (31:39) Claude assists Everett's campaign without an agent swarm (33:09) The specification still starts with human intent (37:54) Write down intent before choosing a language (39:27) Specify the API boundary before accepting software (40:03) Formal verification does not guarantee AI alignment (42:14) Industry-wide AI failures could trigger collective bailouts (44:01) A shared AI layer could disable critical infrastructure (45:20) Why do passenger planes still need human pilots? (46:18) Should autonomous cars keep their steering wheels? (47:05) AI builds keynotes from short DM exchanges (49:56) Rent the intelligence, own the context (52:17) AI spotted cancer before it could do taxes (53:12) Give your agents a panic button (54:06) The five x estimate might be too low (55:20) Agents emailing agents, with three layers of protection (58:30) AI swarm sets 300 new records in five days (1:00:01) Why Nathan gives his AI tools away for free Guests Everett Hildenbrandt — CEO, Runtime Verification (𝕏) Grigore Rosu — Founder, Runtime Verification (𝕏)