AI Platform: Memory, Tools, Guardrails, Observability, Shipping a Production Agent - Dewang Sultania
DataTalks.ClubShow Notes
In this talk, Dewang Sultania, ML Engineer at Netflix and co-author of Designing AI Systems, shares his technical journey from building early inference platforms at Adobe to deploying multimodal AI models at scale. We explore the complete architecture of enterprise AI platforms, focusing on evaluation datasets, tool calling orchestration, and the practical realities of shipping production agents.
You’ll learn about: - The essential components needed to build an enterprise AI platform from scratch - How traditional software engineers can transition into AI platform engineering - The critical technical differences between session management and LLM memory - Strategies for building, versioning, and trusting custom evaluation datasets - Optimizing LLM tool calling and selecting the right performance benchmarks - When to use multi-agent architectures versus single-agent systems - Techniques for overcoming the lost in the middle problem in RAG pipelines
TIMECODES: 00:00 Core Components of Enterprise AI Platforms 11:46 Career Path to AI Platform Engineering 18:33 Session Management Versus LLM Memory Management 26:47 Best Practices for Custom LLM Evaluation Datasets 32:28 LLM Tool Calling Optimization and Benchmark Selection 39:25 Context Window Limits and AI Agent Tool Scaling 45:37 Single-Agent Versus Multi-Agent System Architecture 52:16 LLM-as-a-Judge Alignment and Human Evaluation Strategies 58:11 Solutions for the Lost in the Middle Problem in RAG Systems 01:03:27 Designing AI Systems Book Announcement
This discussion is perfect for software engineers, machine learning practitioners, and technical leaders building real-world AI applications. It offers practical insights for anyone transitioning into AI platform engineering or tasked with scaling LLM features for enterprise production.
Connect with Dewang: - Linkedin - https://www.linkedin.com/in/dewang-sultania/ - Github - https://github.com/iron-stark
Connect with DataTalks.Club: - Join the community - https://datatalks.club/slack.html - Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Check other upcoming events - https://lu.ma/dtc-events - GitHub: https://github.com/DataTalksClub - LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/
More from DataTalks.Club
Tejas Kumar joins Alexey Grigorev to discuss building evaluation datasets, calibrating LLM judges against human preferences, and testing agents beyond their final answers.
In this talk, Suhas Suresha, Senior Machine Learning Engineer at Adobe and co-author of Designing AI Systems, shares his expertise in building large-scale generative AI infrastructure from deploying real-time ECG analysis models at his previous startup, QALY, to architecting enterprise LLM platforms. We explore the massive engineering effort required to take an AI application from a weekend prototype to a scalable production system, and how to build a unified AI platform that eliminates "AI sprawl."
In this talk, Sahil Walia, Senior Technical Architect at Snowflake, shares his extensive data engineering expertise from writing simple SQL queries early in his career to architecting enterprise-scale Apache Iceberg lakehouses today. We explore the evolution of the modern data platform, the mechanics of open lakehouses, and how decoupling storage from compute is transforming data architecture.
In this talk, Philip Christos, Associate at M2VC, shares his expertise in frontier AI investing from his early days as a data analyst to evaluating cutting-edge deep tech startups. We explore how AI agents are reshaping modern work, the realities of venture capital in the LLM era, and why the future of software engineering relies more on high-level systems thinking than writing syntax.You’ll learn about:- How venture capitalists formulate AI investment strategies and evaluate startup pitch decks.- The rapid adoption of AI in software engineering compared to traditional sectors like accounting.- The crucial difference between data progress and algorithmic progress in frontier models.- Why LLMs still struggle with high-level software architecture, business logic, and design "taste."- How AI is accelerating iterations and breakthroughs in classical engineering and scientific research.- What specific deep tech and AI infrastructure ideas are actually attracting VC funding today.TIMECODES:00:00 Venture Capital Career Journey05:14 VC Analyst vs Data Analyst Roles10:13 AI Investment Thesis Strategy16:00 Emotion Analytics and Voice AI Startups22:09 LLM Impact on Venture Capital28:19 Software Engineering AI Adoption34:18 Data vs Algorithmic AI Progress40:15 Classical Engineering AI Acceleration46:12 Software Architecture AI Limitations51:11 Future Software Engineering Careers57:55 Systems Thinking in DevOps1:03:00 Pitching Deep Tech StartupsThis talk is perfect for software engineers, AI founders, and technical product managers who want to understand the evolving tech job market and venture landscape. It provides essential insights for anyone looking to build VC-backed AI startups or future-proof their career as AI coding tools become ubiquitous.Connect with Philip- Linkedin - https://www.linkedin.com/in/philipchristos/- Twitter - https://x.com/phil_christos- Website - https://t.me/stacked_channelConnect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/
In this talk, Paul Domanski, founder of Domanski AI, shares his unconventional journey from film directing and affiliate marketing to becoming a vibecoder running dozens of autonomous AI agents. We explore the realities of building an agentic operating system, how non technical builders can orchestrate complex AI workflows, and the exact strategies used to close high ticket B2B clients using AI generated prototypes.Links:- Headroom: https://github.com/domanski-ai/headroom/You will learn about:- How to build and use custom workspaces like Demox and Tmux setups to orchestrate over 50 concurrent AI agents- Using automated research and rapid AI prototyping to close high ticket B2B sales- Balancing usage across multiple Claude and Codex accounts while navigating API limits and context windows- How to design audit and deploy complex software solutions without a traditional computer science background or deep knowledge of tech stacks- How the skills of a film director like vision stubbornness and delegation perfectly translate to managing autonomous AI teams- Why evaluating LLM outputs is the most critical skill for ensuring reliable products for enterprise clientsTIMECODES:00:00 Welcome and Showcasing Demox08:46 Paul's Background in Film and Digital Marketing16:09 Sales Pipeline and Client Onboarding22:20 Token Efficiency and Open Source Models31:19 Curiosity and the Learning Process37:52 Red Teaming and Code Audits44:51 Building Software Without Knowing the Tech Stack51:02 Parallels Between Film Directing and Managing AI56:32 Using AI for 3D Printing and Personal Projects01:04:34 The Importance of AI EvalsThis conversation is perfect for solopreneurs, agency owners, and non technical builders who want to move beyond basic chat interfaces and start leveraging autonomous AI agents for real world business operations. Whether you are transitioning into AI from marketing, project management, or creative fields, this episode offers a practical blueprint for turning AI curiosity into a scalable business.Connect with Paul:- Linkedin - https://www.linkedin.com/in/paul-domanski-ai/- Twitter - https://x.com/domanski_ai- Github - https://github.com/domanski-ai- Website - https://domanski.ai/Connect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/