The Five-Layer Cake Approach to Scaling AI Without Wasting Money
Agentic Conversations (formally mlops.community)Show Notes
In this episode of Agentic Conversations, we sit down with Ambud Sharma, Principal Engineer at Pinterest, responsible for general technology efficiency, fresh off delivering a controversial keynote on AI infrastructure optimization at scale.
Ambud walks us through his Five Layer Cake framework - a structured approach to driving efficiency across every level of the AI stack, from silicon and hardware procurement to model selection, inference engine design, and governance. We explore how decisions compound across layers to unlock real business growth, and how the wrong choices can lock you into expensive commitments for years.
We stress test the framework against two very different business models: what the stack looks like if you are building the next Cursor, and how it changes entirely if you are building the next YouTube. Along the way we cover hardware immutability, inference engine warm-up costs, GPU occupancy, context switching, quantization trade-offs, model routing, and why experimentation discipline is the only thing that keeps AI infrastructure costs from getting out of control.
We also look at how this framework holds up in the emerging agent era, what changes when agent-to-agent communication becomes the norm, and why agent traffic just passed bot traffic on Cloudflare. The conversation closes on a deceptively simple takeaway: there is no silver bullet, and experimentation at every layer always comes first.
Pinterest: https://about.pinterest.com/
Alex Salkever: https://www.linkedin.com/in/alexsalkever Ambud Sharma: https://www.linkedin.com/in/ambud
Timestamps: [0:00] Introduction and the controversial keynote [2:09] The five-layer cake explained [4:30] Why hardware decisions are irreversible [6:47] Two business models: building Cursor vs YouTube [10:42] Applying the five layers to a YouTube-style company [14:23] Experimentation as the core efficiency method [17:09] Inference stack: context switching and warm-up costs [20:07] Model layer: why changing models breaks everything [24:10] When you should not use an LLM at all [26:41] Governance and routing: right model for the right task [29:20] Horror stories of unchecked token spend [31:37] Experimentation discipline without stifling innovation [34:35] How the five layers change in the agent era [36:05] Agent-to-agent communication and governance complexity [38:27] Core takeaway: experimentation first at every layer
More from Agentic Conversations (formally mlops.community)
What happens when you give 200 engineers who have barely touched AI a coding assistant and a six-month deadline to rebuild their platform? Pepijn Senders, a principal engineer who previously led AI enablement at HelloFresh, has done exactly that, and he chose Sonnet over Opus on purpose.
An AI slips out of its sandbox. A chatbot won’t explain your cat’s stripes. And your next shopping spree might not involve you clicking anything.
What happens to the web when the ones browsing it are no longer people? Sarah Drasner, Area Tech Lead at Google, sits down with us ahead of her Agentic Web keynote at AGNTCon + MCPCon North America. She works through that question from both directions: rebuild the web so agents can use it properly, or build agents smart enough to handle the web as it already exists.
Everyone is racing to build one agent that can do everything. Justin Schroeder, co-founder of Standard Agents, thinks the future looks more like thousands of tiny ones. Ahead of his talk on domain-specific agents at AGNTCon + MCPCon North America, he explains why tightly scoped agents might be the smarter bet, even if it feels like a step back toward the single-purpose machine learning models of a few years ago.
What happens to an MMORPG when half the players never log off? Issac Lee, who runs AI and blockchain experiments at NEXUS, has been testing that idea. His team is filling game worlds with agent players instead of scripted NPCs, so a new title feels alive from day one.