Show Notes
Building AI products isn’t just about clever prompts and orchestration—it’s about knowing if what you’ve built actually works. In this episode, Teresa Torres and Petra Wille dive deep into AI evals: how they’re defined, why they’re essential, and how teams can implement them to ensure product quality.
Teresa shares her journey building her Interview Coach tool and the hard lessons she learned about evals along the way. From golden datasets and synthetic data to error analysis, code-based checks, and LLM-as-judge methods, you’ll walk away with a clearer picture of how to measure and improve AI products over time.
More from All Things Product with Teresa and Petra
Team events are making a comeback. After years of remote work and tight budgets, companies are bringing people together again, and many are looking for experiences that go beyond the usual offsite. So should companies deliberately send their people somewhere strange, uncomfortable, and awe-inspiring, and what actually happens to a team when they do?
You've got twelve people and one hour booked at next week's offsite. Can someone teach your team to do continuous discovery in that hour? Teresa Torres's answer is a flat no — and this episode is about why.
We now record almost everything — every meeting has an AI note-taker, and some people wear glasses that capture every word. So what does that do to our relationships when nothing is forgotten and everything can be fact-checked?
Is innovation the goal — or just a byproduct of solving customer problems well? In this episode of All Things Product, Petra Wille and Teresa Torres unpack one of the most misused words in product: "innovation." Teresa makes the case that chasing novelty for its own sake pushes teams toward complex solutions that don't actually serve customers better, while Petra argues that simplicity and reduction can be their own kind of innovation.
What if the ideas you throw away tell you more about the health of your product process than the ones you ship?