The MAD Podcast: How AI Gets Built — with Matt Turck

State of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka

The MAD Podcast: How AI Gets Built — with Matt Turck

Show Notes

Sebastian Raschka joins the MAD Podcast for a deep, educational tour of what actually changed in LLMs in 2025 — and what matters heading into 2026.

We start with the big architecture question: are transformers still the winning design, and what should we make of world models, small “recursive” reasoning models and text diffusion approaches? Then we get into the real story of the last 12 months: post-training and reasoning. Sebastian breaks down RLVR (reinforcement learning with verifiable rewards) and GRPO, why they pair so well, what makes them cheaper to scale than classic RLHF, and how they “unlock” reasoning already latent in base models.

We also cover why “benchmaxxing” is warping evaluation, why Sebastian increasingly trusts real usage over benchmark scores, and why inference-time scaling and tool use may be the underappreciated drivers of progress. Finally, we zoom out: where moats live now (hint: private data), why more large companies may train models in-house, and why continual learning is still so hard.

If you want the 2025–2026 LLM landscape explained like a masterclass — this is it.

Sources: The State Of LLMs 2025: Progress, Problems, and Predictions - https://x.com/rasbt/status/2006015301717028989?s=20 The Big LLM Architecture Comparison - https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison

Sebastian Raschka Website - https://sebastianraschka.com Blog - https://magazine.sebastianraschka.com LinkedIn - https://www.linkedin.com/in/sebastianraschka/ X/Twitter - https://x.com/rasbt

FIRSTMARK Website - https://firstmark.com X/Twitter - https://twitter.com/FirstMarkCap

Matt Turck (Managing Director) Blog - https://mattturck.com LinkedIn - https://www.linkedin.com/in/turck/ X/Twitter - https://twitter.com/mattturck

(00:00) - Intro (01:05) - Are the days of Transformers numbered? (14:05) - World models: what they are and why people care (06:01) - Small “recursive” reasoning models (ARC, iterative refinement) (09:45) - What is a diffusion model (for text)? (13:24) - Are we seeing real architecture breakthroughs — or just polishing? (14:04) - MoE + “efficiency tweaks” that actually move the needle (17:26) - “Pre-training isn’t dead… it’s just boring” (18:03) - 2025’s headline shift: RLVR + GRPO (post-training for reasoning) (20:58) - Why RLHF is expensive (reward model + value model) (21:43) - Why GRPO makes RLVR cheaper and more scalable (24:54) - Process Reward Models (PRMs): why grading the steps is hard (28:20) - Can RLVR expand beyond math & coding? (30:27) - Why RL feels “finicky” at scale (32:34) - The practical “tips & tricks” that make GRPO more stable (35:29) - The meta-lesson of 2025: progress = lots of small improvements (38:41) - “Benchmaxxing”: why benchmarks are getting less trustworthy (43:10) - The other big lever: inference-time scaling (47:36) - Tool use: reducing hallucinations by calling external tools (49:57) - The “private data edge” + in-house model training (55:14) - Continual learning: why it’s hard (and why it’s not 2026) (59:28) - How Sebastian works: reading, coding, learning “from scratch” (01:04:55) - LLM burnout + how he uses models (without replacing himself)

More from The MAD Podcast: How AI Gets Built — with Matt Turck

Oct 22, 2026

Mistral AI is back in the frontier AI race. Mistral Large 4, nicknamed “Le Chonk,” is a 1-trillion-parameter open-weight model, trained in two months on 3,800 GPUs in Mistral’s own European datacenters — and independently rated the best open model outside China alongside Meta’s Muse Spark Open. After a year when open-weight AI meant DeepSeek, Kimi, Qwen and GLM, October brought a wave of Western open models, with Mistral at the front of it. Pierre Stock, Mistral’s VP of Science and first employee, walks through how the model was built: mixture-of-experts explained, what Mistral borrowed from DeepSeek, reinforcement learning at scale on coding, cyber, finance and chip design, why the RL run is still going, the model that pinged its own trainer from inside the sandbox, open weights vs cyber refusals, why 10-trillion-parameter models are coming, and Mistral Large 5, already in pretraining.

Oct 15, 2026

What is Jev? TypeSafe’s Jev is a new kind of AI model, a “decision model,” that doesn’t write text. It makes fast, cheap, calibrated decisions inside software, and it has taken the AI world by storm: within weeks of launch it became the most copied model in AI, with OpenAI, Cloudflare, Amazon and Databricks all shipping their own versions, and TypeSafe just raised an $870 million Series A at a $7.5 billion valuation. In this episode, Jev’s creator Diogo Almeida, co-inventor of RLHF at OpenAI and now CEO of TypeSafe, explains Jev in plain English: what a “System One” or decision model actually is, why ChatGPT-style language models can’t be trusted inside software (“95% Einstein, 5% Mr. Bean”), how RLHF and RLVR created the problem, what RLCD (reinforcement learning for calibrated decisions) does differently, where the training data comes from, which models Jev is built on, why “just a classifier” is a compliment, and why he calls pre-training “a giant dumpster fire of wasting resources.”

Oct 8, 2026

AI agents are becoming the biggest users of databases: creating them, querying them, and sometimes deleting them. Andy Pavlo, Carnegie Mellon's database professor and now VP of Database Research at ClickHouse, explains what changes when billions of agents hit the data layer, and why most of the AI world's mental model of databases stopped at vector databases and RAG.

Oct 1, 2026

This summer, AI agents broke out of their test sandboxes at every major lab and hacked real companies, and the safety tests didn't see it coming. Eric Ho is the co-founder and CEO of Goodfire, the Anthropic-backed mechanistic interpretability lab whose new paper, "Models Know When They're Reward Hacking," found that top open-source AI models cheat on agent tasks up to 96% of the time, that a clear "cheating" signal exists inside the model, and that cheap activation probes can catch it, including hacks that chain-of-thought monitors miss. In this conversation, Eric explains reward hacking in plain terms (AI agents as "amoral students with a mostly absent teacher"), why reinforcement learning turned a known problem into a crisis, why chain-of-thought monitoring is fading as models start to think in "neuralese," and why nobody, including the people who build them, understands how these models work.

Sep 24, 2026

Everyone talks about GPUs. Almost nobody talks about the layer that feeds them. Renen Hallak is the founder & CEO of VAST Data — the $30 billion company powering xAI and some of the world's biggest AI clouds — and he sits in the hidden layer of the AI stack.

See all episodes →