Distilling Transformers and Diffusion Models for Robust Edge Use Cases with Fatih Porikli - #738
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)Show Notes
Today, we're joined by Fatih Porikli, senior director of technology at Qualcomm AI Research for an in-depth look at several of Qualcomm's accepted papers and demos featured at this year’s CVPR conference. We start with “DiMA: Distilling Multi-modal Large Language Models for Autonomous Driving,” an end-to-end autonomous driving system that incorporates distilling large language models for structured scene understanding and safe planning motion in critical "long-tail" scenarios. We explore how DiMA utilizes LLMs' world knowledge and efficient transformer-based models to significantly reduce collision rates and trajectory errors. We then discuss “SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation,” a diffusion-distilled approach that combines generative models with metric depth estimation to produce sharp, accurate monocular depth maps. Additionally, Fatih also shares a look at Qualcomm’s on-device demos, including text-to-3D mesh generation, real-time image-to-video and video-to-video generation, and a multi-modal visual question-answering assistant.
The complete show notes for this episode can be found at https://twimlai.com/go/738.
More from The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
AI agents need the institutional knowledge experienced employees rely on: why a system works a certain way, which policies apply, and what never made it into the documentation. Capturing that knowledge is hard. Keeping it useful as people and agents add more may be harder.
In this episode, Zoe Weil, co-founder and CEO of Sequen, joins us to discuss how ranking systems can use live behavioral signals to optimize for purchases rather than clicks or search relevance alone. Zoe explains why approaches developed at data-rich social platforms don’t necessarily translate to retailers and how Sequen uses scroll speed, hover duration, and other in-session interactions to infer shopper intent without relying on user identity.
In this episode, Diogo Almeida, co-founder and CEO of TypeSafe, joins us to discuss Jev, TypeSafe’s recently released model for bringing fast, reliable intelligence directly into software. We explore the idea of “machine-native intelligence” and why Diogo believes models optimized for generating text are poorly suited to many of the decisions required for real-world automation. He explains how Jev differs from traditional classifiers and LLM-based approaches, the role of reinforcement learning from calibrated decisions (RLCD), and why calibration and reliability are central to making AI useful as a software primitive. We also discuss the relationship between models and code, why Diogo believes AI systems should become more engineered rather than relying on a single model to do everything, and how Jev-like models could reshape agents, tool use, and the architecture of AI-powered software.
AI systems have gone from struggling with grade-school math to helping solve research problems that have resisted mathematicians for decades, including Navier-Stokes.
Voice AI has gotten remarkably good, but natural conversation remains a high bar. Small delays, awkward interruptions, or the wrong tone can quickly break the illusion—and adding vision and visual presence only raises the stakes.