
The Biggest Chip Ever Built — Why OpenAI Runs On It | Cerebras CEO Andrew Feldman
The MAD Podcast: How AI Gets Built — with Matt TurckShow Notes
AI is no longer just a race to train smarter models. As AI moves into production, the bottleneck is increasingly inference: how fast models can generate tokens, use tools, reason, verify, and act. In this episode of the MAD Podcast, Matt Turck sits down with Andrew Feldman, co-founder and CEO of Cerebras, to explain why fast inference may define the next era of AI.
Cerebras is known for building a chip the size of a silicon wafer. But this conversation is not just about one company or one chip. It is a deep dive into the AI infrastructure stack: GPUs, ASICs, memory, HBM, SRAM, data centers, power, TSMC, AWS, OpenAI, agents, reasoning models, and why speed changes what AI products can become. Andrew explains why “tokens per second per user” matters, why generating a single word can require moving the equivalent of 100 HD movies through memory, why agents amplify latency, why GPUs struggle with certain inference workloads, and why fast AI may eventually reshape SaaS itself.
This is a reference conversation on fast inference, AI chips, and the next compute bottleneck.
(00:00) Cold open & Intro (01:31) Why speed became the AI bottleneck (02:32) Tokens per second per user, explained (03:16) AI’s broadband moment and the Netflix analogy (04:35) The AI chip landscape: GPUs, TPUs, Trainium, ASICs (06:36) What is an ASIC? (08:08) Nvidia, Groq, and the fast inference war (09:16) OpenAI, Broadcom, and specialized silicon (12:10) China, power, and sovereign AI infrastructure (15:05) Is the AI infrastructure boom a bubble? (18:56) The hidden bottlenecks: HBM, CoWoS, and 3nm (22:57) Why agents are creating CPU demand (25:36) Andrew Feldman’s path from SeaMicro to Cerebras (26:13) Why Cerebras bet on AI in 2016 (31:14) SRAM vs. HBM: why inference is a memory problem (33:19) What wafer-scale computing actually means (34:28) The deep-tech “Everest” problem (36:07) The moment the first Cerebras system worked (36:49) Ringing the bell and surviving deep tech (39:08) How a giant chip handles failure (41:22) Why GPUs struggle with decode (42:17) Prefill vs. decode explained (44:01) The “100 HD movies” problem in AI inference (45:04) How fast inference changes RL and training (48:08) Reasoning models and why they cost more compute (50:08) Verification, guardrails, and small models checking big models (52:37) Multimodal AI and the path to video (53:51) Cerebras’ business model: hardware, cloud, and API (55:14) OpenAI’s 750MW inference deal (55:36) Why data centers are measured in megawatts (58:01) AWS Trainium + Cerebras decode (59:29) Fast tokens as a cloud product (01:00:52) Is CUDA still a moat? (01:03:53) How TSMC helped Cerebras build the giant chip (01:07:41) Why nobody cared in 2020 (01:08:15) Why chip supply chains are hard to diversify (01:09:54) Why today’s AI models will be the worst you ever use (01:10:38) What fast AI could do to SaaS
More from The MAD Podcast: How AI Gets Built — with Matt Turck
Mistral AI is back in the frontier AI race. Mistral Large 4, nicknamed “Le Chonk,” is a 1-trillion-parameter open-weight model, trained in two months on 3,800 GPUs in Mistral’s own European datacenters — and independently rated the best open model outside China alongside Meta’s Muse Spark Open. After a year when open-weight AI meant DeepSeek, Kimi, Qwen and GLM, October brought a wave of Western open models, with Mistral at the front of it. Pierre Stock, Mistral’s VP of Science and first employee, walks through how the model was built: mixture-of-experts explained, what Mistral borrowed from DeepSeek, reinforcement learning at scale on coding, cyber, finance and chip design, why the RL run is still going, the model that pinged its own trainer from inside the sandbox, open weights vs cyber refusals, why 10-trillion-parameter models are coming, and Mistral Large 5, already in pretraining.
What is Jev? TypeSafe’s Jev is a new kind of AI model, a “decision model,” that doesn’t write text. It makes fast, cheap, calibrated decisions inside software, and it has taken the AI world by storm: within weeks of launch it became the most copied model in AI, with OpenAI, Cloudflare, Amazon and Databricks all shipping their own versions, and TypeSafe just raised an $870 million Series A at a $7.5 billion valuation. In this episode, Jev’s creator Diogo Almeida, co-inventor of RLHF at OpenAI and now CEO of TypeSafe, explains Jev in plain English: what a “System One” or decision model actually is, why ChatGPT-style language models can’t be trusted inside software (“95% Einstein, 5% Mr. Bean”), how RLHF and RLVR created the problem, what RLCD (reinforcement learning for calibrated decisions) does differently, where the training data comes from, which models Jev is built on, why “just a classifier” is a compliment, and why he calls pre-training “a giant dumpster fire of wasting resources.”
AI agents are becoming the biggest users of databases: creating them, querying them, and sometimes deleting them. Andy Pavlo, Carnegie Mellon's database professor and now VP of Database Research at ClickHouse, explains what changes when billions of agents hit the data layer, and why most of the AI world's mental model of databases stopped at vector databases and RAG.
This summer, AI agents broke out of their test sandboxes at every major lab and hacked real companies, and the safety tests didn't see it coming. Eric Ho is the co-founder and CEO of Goodfire, the Anthropic-backed mechanistic interpretability lab whose new paper, "Models Know When They're Reward Hacking," found that top open-source AI models cheat on agent tasks up to 96% of the time, that a clear "cheating" signal exists inside the model, and that cheap activation probes can catch it, including hacks that chain-of-thought monitors miss. In this conversation, Eric explains reward hacking in plain terms (AI agents as "amoral students with a mostly absent teacher"), why reinforcement learning turned a known problem into a crisis, why chain-of-thought monitoring is fading as models start to think in "neuralese," and why nobody, including the people who build them, understands how these models work.
Everyone talks about GPUs. Almost nobody talks about the layer that feeds them. Renen Hallak is the founder & CEO of VAST Data — the $30 billion company powering xAI and some of the world's biggest AI clouds — and he sits in the hidden layer of the AI stack.