Show Notes
Summary In this episode Raj Shukla, CTO of SymphonyAI, explores what it really takes to build self‑improving AI systems that work in production. Raj unpacks how agentic systems interact with real-world environments, the feedback loops that enable continuous learning, and why intelligent memory layers often provide the most practical middle ground between prompt tweaks and full Reinforcement Learning. He discusses the architecture needed around models - data ingestion, sensors, action layers, sandboxes, RBAC, and agent lifecycle management - to reach enterprise-grade reliability, as well as the policy alignment steps required for regulated domains like financial crime. Raj shares hard-won lessons on tool use evolution (from bespoke tools to filesystem and Unix primitives), dynamic code-writing subagents, model version brittleness, and how organizations can standardize process and entity graphs to accelerate time-to-value. He also dives into pitfalls such as policy gaps and tribal knowledge, strategies for staged rollouts and monitoring, and where small models and cost optimization make sense. Raj closes with a vision for bringing RL-style improvement to enterprises without requiring a research team - letting businesses own the reasoning and memory layers that truly differentiate their AI systems.
Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management If you lead a data team, you know this pain: Every department needs dashboards, reports, custom views, and they all come to you. So you're either the bottleneck slowing everyone down, or you're spending all your time building one-off tools instead of doing actual data work. Retool gives you a way to break that cycle. Their platform lets people build custom apps on your company data—while keeping it all secure. Type a prompt like 'Build me a self-service reporting tool that lets teams query customer metrics from Databricks—and they get a production-ready app with the permissions and governance built in. They can self-serve, and you get your time back. It's data democratization without the chaos. Check out Retool at dataengineeringpodcast.com/retool today and see how other data teams are scaling self-service. Because let's be honest—we all need to Retool how we handle data requests. Your host is Tobias Macey, and today I’m interviewing Raj Shukla about building self-improving AI systems — and how they enable AI scalability in real production environments.
Interview
Introduction How did you get involved in AI/ML? Can you start by outlining what actually improves over time in a self-improving AI system? How is that different from simply improving a model or an agent?
How would you differentiate between an agent/agentic system vs. a self-improving system? One of the components that are becoming common in agentic architectures is a "memory" layer. What are some of the ways that contributes to a self-improvement feedback loop? In what ways are memory layers insufficient for a generalized self-improvement capability?
For engineering and technology leaders, what are the key architectural and operational steps you recommend to build AI that can move from pilots into scalable, production systems? One of the perennial challenges for technology leaders is how to build AI systems that scale over time. How has AI changed the way you think about long-term advantage? How do self-improvement feedback loops contribute to AI scalability in real systems? What are some of the other key elements necessary to build a truly evolutionary AI system? What are the hidden costs of building these AI systems that teams should know before starting? I’m talking about enterprise who are deploying AI into their internal mission-critical workflows. What are the most interesting, innovative, or unexpected ways that you have seen self-improving AI systems implemented? What are the most interesting, unexpected, or challenging lessons that you have learned
More from Data Engineering Podcast
Summary In this episode, I spoke with Keerti Kulkarni about the data and AI systems behind physical logistics automation at Amazon. She explained how her team supports dock-door operations across more than 100 North American facilities by turning telemetry from physical workflows into actionable operational intelligence. We discussed the end-to-end data path from CloudWatch, Firehose, and S3 into Redshift and AI-powered analysis, along with the realities of handling petabyte-scale event streams, late and duplicate data, reconciliation, cost controls, and the need to treat physical telemetry as a first-class signal rather than noisy exhaust. Kirti also shared how these systems are designed to protect worker privacy while still surfacing the right information for safety and operational decisions. We also dug into the role of LLMs and agentic workflows in helping operations teams diagnose failures quickly through conversational interfaces. Kirti described how AI-generated summaries help identify root causes such as authorization problems, hardware issues, or connectivity failures, while human feedback loops, structured outputs, and strict vocabulary controls help manage hallucination risk. Beyond troubleshooting, we explored how the platform is evolving toward proactive maintenance, training insights, per-site cost attribution, and broader modular agent architectures that could be shared across logistics teams. It’s a detailed look at what it takes to combine data engineering, real-time operations, and AI in a large-scale physical environment.
Summary In this episode Christopher Doidge talks about his Agile Ledger Architecture (ALA) approach to data warehousing and how it aims to reduce data debt while shortening the path from raw data to trustworthy business insight. Christopher explained that ALA is not a replacement for existing warehouse patterns like medallion architecture, star schemas, or other modeling approaches, but a complementary discipline focused on pushing business definitions upstream, enforcing cleaner ledger-style transformations, and producing gold-layer tables that stakeholders can actually use without relying on analysts to repeatedly rebuild the same logic. He also discussed his “15-minute litmus test” for time-to-insight, the importance of durable documentation through a data dictionary, and why undocumented business logic living in analyst scripts is often one of the biggest hidden forms of data debt. Overall, this was a thoughtful conversation about designing warehouse systems that serve not just analysts, but the broader business as well. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Today’s episode is sponsored by Parallel - where agents find answers. Most engineers today closely follow new model releases, but don’t pay attention to their agent’s most important tool: web search. Parallel develops enterprise-grade infrastructure for agents to retrieve high-quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto-optimal quality, cost, and speed. Whether you work on voice agents that need 200 millisecond latency, chat bots that balance speed, depth, and quality, or long-horizon agents to do thorough, overnight research for you, Parallel is a single platform for all your agentic research. Get started for free at dataengineeringpodcast.com/parallel Your host is Tobias Macey and today I'm interviewing Christopher Doidge about a data warehousing approach called Agile Ledger Architecture that is designed to drive down data debt Interview Introduction How did you get involved in the area of data management? Can you describe what the agile ledger architecture is and the story behind it? There are numerous patterns and practices that have been developed for data warehousing over the past 40 years. How does the agile ledger architecture fit in that ecosystem? (e.g. is it compatible with Kimball, Inmon, Data Vault, Anchor Modeling, etc.?) What are some examples of the types of debt that accumulate in current approaches to warehouse implementation, and the impact that it has on the utility of that asset? Digging into the architecture itself, what are the core principles that it is built on? What are the technologies or practices that it is best suited to? (e.g. event streams, lakehouse, ELT workflows, etc.) For someone who has already invested a substantial amount of effort into building a warehouse, what does adoption of the agile ledger architecture look like? Once you have started that adoption, what are the technical controls that you can put in place to ensure that this architecture is maintained and doesn't regress or get sidestepped in another portion of the warehouse? As LLMs and agents grow to become the predominant consumers (and often producers) of a warehouse, what are the benefits that the agile ledger architecture provides to ensure appropriate context and grounding to produce useful insights and accurate answers? What are the most interesting, innovative, or unexpected ways that you have seen the agile ledger architecture used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on data warehouse design and implementation? When is the agile ledger architecture the wrong choice? What do you have planned for the future of this architecture? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for
Summary In this episode Soham Mazumdar, co-founder and CEO of Wisdom.ai, talks about what “context” really means in data engineering and AI systems. He explores why context has become such an overloaded term, spanning everything from semantic layers and data catalogs to tribal knowledge, query logs, dashboards, and even agent memory. Soham explained that the big shift is that context is no longer being prepared primarily for human analysts, but for LLMs and agents that can’t reliably fill in missing gaps on their own. That change raises the bar for how context is represented, validated, benchmarked, and maintained so that AI systems can produce trustworthy outcomes.
Summary In this episode Yetunde Dada discusses Otto, Astronomer’s AI agent for Airflow, and the broader challenge of making agentic tooling actually useful for data engineers. She explored why generic coding assistants often fall short in data workflows, how Otto adds the missing context around Airflow, Astro, upgrades, and troubleshooting, and why Astronomer focused first on high-leverage use cases such as DAG authoring, investigation of pipeline failures, version migrations, and legacy scheduler modernization. She also discussed the practical realities of introducing agents into engineering teams: model choice, security boundaries, vendor lock-in concerns, validation of generated code, and the need for agents to fit into existing workflows rather than forcing users into new ones. Overall, this conversation offers a detailed look at how specialized AI agents can support data engineers today, and where Astronomer is headed next with a vision for self-healing pipelines that keep humans in control while automating more of the operational burden.