Show Notes
Are your AI agents unreliable? In this guide, we reveal a professional system for AI evals to help you build and ship better AI products, faster. Learn how to systematically test LLM performance, evaluate complex tool use, and improve multi-turn conversations. We break down the exact process for building a high-quality eval dataset, using milestones and minefields to track agent behaviour, and how to properly use an LLM as a judge without compromising quality. Stop guessing and start making real, measurable improvements to your AI today.
Check out Quotient AI https://www.quotientai.co/
Sign up for A.I. coaching for professionals at: https://www.anetic.co
Get FREE AI tools pip install tool-use-ai
Connect with us https://x.com/ToolUseAI https://x.com/MikeBirdTech https://x.com/freddie_v4
00:00:00 - intro 00:02:54 - Why You Need AI Evals 00:06:13 - How to Evaluate AI Agent Tool Use 00:29:24 - The Process for Building Your First Eval Dataset 00:42:44 - Using an LLM as a Judge The Right Way
Subscribe for more insights on AI tools, productivity, and AI evals. Tool Use is a weekly conversation with AI experts brought to you by Anetic.
More from Tool Use - AI Conversations
Check out www.ToolUsePodcast.com Are AI web browsers falling short of the hype? Enter Rover, a revolutionary open-source AI agent that unlocks full autonomous workflows directly on your website with just a single script tag. In this episode, Arjun and Bhavani from rtrvr return to dive deep into how Rover uses a unique DOM-only approach to navigate, click, and type with subsecond latency. This innovative method completely bypasses slow screenshot-based models and outdated playwright or puppeteer scripts. We discuss the engineering secrets behind building smart DOM trees, why they crushed web benchmarks, and how Rover provides a powerful alternative to Google's Web MCP by keeping users engaged on your site instead of handing traffic over to external AI tools. Plus, learn how you can trigger complex agentic workflows via simple URL queries and why open-sourcing this technology is a massive win for the developer community. Rover links:https://x.com/rtrvrairover.rtrvr.ai Github: https://github.com/rtrvr-ai/rover Rover Deep dive: https://www.rtrvr.ai/blog/10-billion-proof-point-every-website-needs-ai-agent Benchmark: https://www.rtrvr.ai/blog/web-bench-results Connect withus https://x.com/ToolUsePodcast https://x.com/MikeBirdTech https://x.com/rtrvrai 00:00:00 - Intro 00:03:05 - The DOM-Only Approach vs Screenshots for AI Agents 00:06:00 - Parsing HTML vs Markdown for Reliable LLM Data 00:11:47 - Handling Modals, iFrames, and Canvas Elements 00:15:32 - Implementing AI Guardrails and Extracting User Intent Data 00:28:04 - Rover vs Google Web MCP for Website Automation 00:31:25 - Triggering Autonomous AI Workflows via URL Queries Subscribe for more insights on AI tools, productivity, and web automation. Join the Tool Use Discord: https://discord.gg/PnEGyXpjaX Tool Use is a weekly conversation with the top AI experts.
Join the Tool Use Discord: https://discord.gg/PnEGyXpjaX
Join the Tool Use Discord: https://discord.gg/PnEGyXpjaX
Join the Tool Use Discord: https://discord.gg/PnEGyXpjaX