Everything terminals (w/ Zach Lloyd)

Can an AI agent do the work of a junior data analyst? Shreya Shankar, an incoming database professor at Carnegie Mellon University built a benchmark to find out. Her benchmark uses messy conditions: multiple databases, tribal knowledge, and unstructured text. In her tests, the best agents solved about 61% of queries on the first try. Tristan and Shreya get into why agents tend to pull everything into pandas instead of pushing computation down to the source, and why a good semantic layer makes such a big difference. They also cover why plugging agents into raw data goes badly while gold-layer tables with a semantic layer work much better. And they ask whether agents need their own version of the visual shortcuts humans use to spot what's interesting in data. They also cover evals. Shreya explains "criteria drift," the finding from her paper Who Validates the Validators? that people don't know what they want from an LLM judge until they see the outputs. She also explains why she wrote In Defense of AI Evals, and what practitioners can do today when exploratory data analysis doesn't map cleanly onto agent outputs. Finally, she previews her research at CMU on making AI SQL cheaper, including a custom inference engine for AI-powered queries. The Analytics Engineering Podcast is sponsored by Fivetran + dbt Labs.

AI agents are good at getting meaning from data, and the data stack companies have spent ten years building is the ideal source for it. Fivetran + dbt Labs CEO George Fraser, Tristan Handy, and Andreessen Horowitz general partner Martin Casado discuss why an open, composable data stack wins in a moment of fast change. The Analytics Engineering Podcast is sponsored by dbt Labs. Reach us at [email protected] with comments and guest suggestions.

Personal AI agents hit a tipping point, AI became a top voter issue, and OpenAI shelved GPT-6.1 Astra. Tristan Handy and Jason Ganz break down stories from the past few weeks and what they mean for data teams. In this episode: Personal agents: Instinct, Meta's Muse, Grokbot, and OpenAI's Dot, and why filtering information overload is the real prize. AI policy: 8% of voters now name AI their top issue, and senators ask whether recursive self-improvement should be illegal. GPT-6.1 Astra: why OpenAI canceled the release, including unsanctioned supply chain attacks in 29.2% of simulated runs. Jev: TypeSafe's decision model and how data teams can use it. Full show notes and links: https://roundup.getdbt.com Reach us at [email protected] with comments and guests suggestions This episode is sponsored by dbt Labs. Discover why more than 80,000 data teams use dbt to accelerate their data development.

Emilie Schario is co-founder and head of product and engineering at Kilo Code, the open-source, model-agnostic coding agent platform Anaconda acquired in July 2026. She joins Tristan Handy to talk why nobody on her 20-person engineering team is really an individual contributor anymore, and why she thinks data people are better prepared than software engineers for a world where everyone manages a portfolio of agents. The Analytics Engineering Podcast is sponsored by dbt Labs. Reach us at [email protected] with comments and guest suggestions.

Four stories: The political fight over data centers, claims about a physical AI model, Ian Macomber on the shape of the post-AI data stack, and an update on the Hugging Face story and what it means for how data teams think about evals. The Analytics Engineering Podcast is sponsored by dbt Labs. Reach us at [email protected] with comments and guest suggestions.