Show Notes
I got up early to record an Opus 5.5 review. Then Anthropic and OpenAI dropped new models on the same morning, and I decided to do something I’d never done before: take the How I AI bench live. I put GPT-6 Astra, GPT-6 Sol, Claude Opus 5.5, and more through the work I actually care about: emails, PRDs, frontend prototypes, backend work, long-running agents, SVGs, and video editing. I scored the outputs without knowing which model made them, so you get to watch me make predictions, change my mind, and reveal my own very inconsistent taste. Astra won my heart. Opus 5.5 won my week. Sol still has me split. There’s a creative result I got completely wrong, an LLM judge that disagreed with me, and a return to Barbie Bench: the 3D fashion game that keeps reminding me how far we have to go. The hands are tragic. AGI has not arrived.
What you’ll learn: How I run the How I AI bench blind, and what gets an output a bad score before I even know which model made it Why Astra won my heart while Opus 5.5 might be overall strongest, especially for long-running agents and B2B frontend Where Sol still wins me over on clear writing, readable PRDs, and price The character SVG results that completely overturned my prediction about Anthropic What happened when I asked these models to edit video, and why I think skills explain part of the disappointment Why an LLM judge disagreed with my rankings, and what it was rewarding that I wasn’t — In this episode, we cover: (00:00) LIVE setup and new model launches (01:30) What’s new in Opus 5.5, Sol, and Luna (04:11) Guardrails, personality, and speed (09:00) The How I AI bench and blind evaluation process (11:31) Email and personal-productivity results (13:50) Frontend prototype vibe checks (24:10) Backend, agent personality, and long-running tasks (28:25) SVG illustration test (29:48) AI video-editing results (30:43) Predictions before the reveal (31:20) Barbie Bench: the 3D fashion-game test (34:17) Results: Astra, Sol, and Opus 5.5 (35:04) Writing clarity and creative surprises (36:51) Why the LLM judge disagreed with me (37:24) What each model is actually best for — Tools referenced: • Claude Opus 5.5: https://www.anthropic.com/claude-opus-5-5 • GPT-6 Sol and Luna: https://openai.com/index/introducing-gpt-6-sol-and-luna/ • Codex (OpenAI): https://openai.com/codex — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].
More from How I AI
Anthropic's new personal agent, Claude Duet, lives on your phone, inherits all your Claude plugins and connectors, and is pitched at small business owners and busy professionals. It's the most polished agent launch I've seen: thoughtful onboarding, task lists, first-class artifacts, and Slack-style channels. But polish isn't enough when you're competing with every other personal agent for attention. I put it to work on my inbox and Slack, my sales pipeline, a Google Docs sales deck, a launch video, and finding Warriors tickets for my kids, and I'll walk you through what it nails, where it stalls, and what Anthropic needs to fix before it can become anyone's number one agent.
Brooke Lacey has been making videos since before Facebook had video, and she built a TikTok audience of more than 290,000 by learning to cut short-form videos herself. With over 25 years in tech, she now builds her own apps, including Zosha, a free mobile app for learning AI. In this episode, Brooke walks me through the Claude Code, FFmpeg, and Whisper workflow that edits her TikToks in about 15 seconds, how she mines her comments for new scripts, and how she runs a weekly short-form operation with no editor.
I put Meta’s new Muse Wonder (and Wonder Full) frontier models through the How I AI bench, which covers nearly 20 tasks across prototyping, agentic workflows, coding, writing, and creative work. I scored everything with an LLM-as-a-judge pass and my own voice notes, then pulled together the first-look results you see here.
Alex Cornell is a design director at Meta, where he leads design on Muse, Meta’s consumer AI agent. He came to the role with a long background in software product design, including time as a founder, and has spent the past year building one of the most thoughtful agent interfaces in the consumer AI space. Muse ships with a custom avatar, a three-layer status display, a personalized ideas surface, and a generative feed. Alex has been sharing the reasoning behind each of those design decisions publicly on Threads.
Kath Korevec is a member of the Product staff at OpenAI working on Codex, and she spent over a year building and using ChatGPT Sites internally before its public launch. She’s been on the front lines of shipping Plugin Insights, MCP plugin hosting, and the connector ecosystem, which now includes around 60 integrations.