Avoiding ‘team debt’ by building small and smart (w/ Matt Cook)

Companies now use an average of 12 models per month, yet the biggest hidden variable isn't which model you pick—it's the inference provider serving it and whether your cache hit rate stays above 85%. Peter Walker from OpenRouter explains how the same open-weight model can have wildly different cost and performance depending on the provider's caching policy, latency tail, and whether a stealth provider is reselling your prompts. The conversation covers why agentic workloads broke cache economics in February 2026, how ZDR policies vary even among "zero data retention" claims, and why the frontier labs are now competing across price and intelligence spectrums simultaneously. For engineers choosing production models, this is a practical map of the dimensions that actually move the bill. The inference provider layer—not just the model layer—now has 200+ competitors, and provider choice directly determines effective cost, latency, and data sovereignty. Cache hit rate is the dominant cost lever for agentic workloads: a cache miss can cost 50-100x more per token, and median agentic cache rates have climbed from ~50% to ~85% in 2026. Most organisations use a 'hub and spoke' model architecture with one 'daily driver' taking the majority of tokens and spend, plus specialised models for departments like customer support or sales. Zero data retention claims vary in trustworthiness by jurisdiction and provider size; some stealth models on OpenRouter are free specifically because they are not ZDR. The gap between frontier and open-weight models is widening in raw capability but narrowing in perceived value, as daily tasks saturate below frontier intelligence. Reasoning-only models and jagged intelligence are driving demand for fast decision models like Jev and OpenAI's decision model for latency-critical paths. 00:00 Introduction 00:38 The New Complexity 04:07 Provider Layer Competition 06:09 Choosing a Provider 08:42 Data Retention Realities 12:39 Model Portfolios 15:23 Latency and Reasoning 18:29 Cache Economics 23:35 Cost-Performance Sweet Spot 26:24 What Comes Next 29:13 Open Weight vs Frontier 32:02 Specialisation and Product 34:14 Where to Follow Peter OpenRouter: https://openrouter.ai OpenRouter Data: https://openrouter.ai/data DeepSeek: https://www.deepseek.com Anthropic: https://www.anthropic.com OpenAI: https://openai.com Google AI: https://ai.google Cerebras: https://www.cerebras.net CoreWeave: https://www.coreweave.com ElevenLabs: https://elevenlabs.io Artificial Analysis: https://artificialanalysis.ai Groq: https://groq.com Peter Walker is Head of Insights at OpenRouter, where he analyses global model usage patterns across the inference layer. This episode is presented by SparkLaunch and Dam Secure. SparkLaunch (https://sparklaunch.io/wizardry): Applied Wizardry listeners get an extra 20% off. Dam Secure (https://damsecure.ai/wizardry): Applied Wizardry listeners get their first month free. Justin McArthur, Executive Producer and Video Editor: https://www.linkedin.com/in/justin-mcarthur/ Steph Hefferan, Editor: https://www.linkedin.com/in/steph-heff/

Follow Applied Wizardry on its new feed to keep getting every episode.

After nearly 300 episodes of The Startup Podcast, Chris Saad is bringing everything he's learned about building world-class products and product strategy into a single tool: ScaleStudio. In what may be our final episode (unless a white knight sponsor appears), Chris walks Yaniv through the product that could fundamentally change how founders approach strategy; from problem statement all the way to positioning, roadmap sequencing, and org design.

A few weeks ago, Rob Snyder came on the pod to introduce his PULL framework for identifying real product-market fit. Now, Yaniv turns the tables and subjects his own startup, Vera, to the same scrutiny.

This is a remastered Replay of one of our most popular episodes, "What Startups Can Learn From Poker", which was originally published on 21 December 2022. Enjoy!