Rethinking Pre-Training for Agentic AI with Aakanksha Chowdhery - #759
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)Show Notes
Today, we're joined by Aakanksha Chowdhery, member of technical staff at Reflection, to explore the fundamental shifts required to build true agentic AI. While the industry has largely focused on post-training techniques to improve reasoning, Aakanksha draws on her experience leading pre-training efforts for Google’s PaLM and early Gemini models to argue that pre-training itself must be rethought to move beyond static benchmarks. We explore the limitations of next-token prediction for multi-step workflows and examine how attention mechanisms, loss objectives, and training data must evolve to support long-form reasoning and planning. Aakanksha shares insights on the difference between context retrieval and actual reasoning, the importance of "trajectory" training data, and why scaling remains essential for discovering emergent agentic capabilities like error recovery and dynamic tool learning.
The complete show notes for this episode can be found at https://twimlai.com/go/759.
More from The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
AI agents need the institutional knowledge experienced employees rely on: why a system works a certain way, which policies apply, and what never made it into the documentation. Capturing that knowledge is hard. Keeping it useful as people and agents add more may be harder.
In this episode, Zoe Weil, co-founder and CEO of Sequen, joins us to discuss how ranking systems can use live behavioral signals to optimize for purchases rather than clicks or search relevance alone. Zoe explains why approaches developed at data-rich social platforms don’t necessarily translate to retailers and how Sequen uses scroll speed, hover duration, and other in-session interactions to infer shopper intent without relying on user identity.
In this episode, Diogo Almeida, co-founder and CEO of TypeSafe, joins us to discuss Jev, TypeSafe’s recently released model for bringing fast, reliable intelligence directly into software. We explore the idea of “machine-native intelligence” and why Diogo believes models optimized for generating text are poorly suited to many of the decisions required for real-world automation. He explains how Jev differs from traditional classifiers and LLM-based approaches, the role of reinforcement learning from calibrated decisions (RLCD), and why calibration and reliability are central to making AI useful as a software primitive. We also discuss the relationship between models and code, why Diogo believes AI systems should become more engineered rather than relying on a single model to do everything, and how Jev-like models could reshape agents, tool use, and the architecture of AI-powered software.
AI systems have gone from struggling with grade-school math to helping solve research problems that have resisted mathematicians for decades, including Navier-Stokes.
Voice AI has gotten remarkably good, but natural conversation remains a high bar. Small delays, awkward interruptions, or the wrong tone can quickly break the illusion—and adding vision and visual presence only raises the stakes.