Observability in the AI Era with New Relic's Nic Benders

We're hitting pause for the summer on new episodes of the Platform Engineering Podcast, but don’t worry, we’ve got some great episodes in the feed to keep you entertained in the meantime. If you’ve got suggestions for guests or topics, please reach out — I’m always eager to annoy people into joining the conversation! Can’t wait to reconnect with you all soon!

Network calls fail in ways function calls never do - and once a monolith becomes microservices, reliability problems show up fast: retries amplify load, latency spikes cascade, and “what talks to what?” becomes hard to answer. William Morgan, co-creator of Linkerd and the person who coined “service mesh,” breaks down what service meshes actually solve for platform teams running Kubernetes at scale. The conversation focuses on practical outcomes: improving reliability between services, getting uniform observability without rewriting every app, and handling gaps Kubernetes doesn’t cover well - like gRPC/HTTP2 load balancing and cross-environment communication. Key topics Why reliability is the first “microservices tax” (timeouts, retries, backoff, cascading failure) What Kubernetes does not solve at the networking layer—and where a service mesh fits gRPC/HTTP2 load balancing problems and why L4 balancing can fall short Service-to-service visibility: understanding traffic flows and performance without per-app instrumentation Cost and resilience tradeoffs with multi-AZ Kubernetes on AWS (and how zonal-aware balancing can help) Whether developers should ever need to interact with service mesh configuration Where zero trust and policy controls belong: platform guardrails vs application ownership

When code gets cheaper to produce, feedback becomes the limiting factor - CI, reviews, and the handoffs between tools can quietly slow everything down. Rob Zuber breaks down what platform engineers are seeing as teams adopt AI-assisted development: more branch builds, new failure modes, and growing pressure to shorten the loop between “change made” and “change validated.” He focuses on how CI can evolve from a human-first dashboard into a system that agents can interact with directly through APIs, CLIs, and MCP-style interfaces - so fixes can happen faster and with less waiting on manual triage. Along the way, Rob and Cory dig into practical questions engineering leaders are wrestling with: how PR review becomes the next major bottleneck, what “agent experience” means in a delivery pipeline, why speed isn’t only about faster compute (it’s also about doing less unnecessary work), and how teams can share learnings so “agentic velocity” doesn’t only benefit a few power users. If you’re building or running the systems that ship software, this is a clear look at where CI fits in an AI-accelerated workflow, and what needs to change to keep delivery safe, fast, and sustainable.

A lot of infrastructure and automation fails for ordinary reasons: rate limits, flaky networks, partial permissions, long-running jobs, and retries that vanish when the process restarts. Durable execution is a way to design systems that keep going anyway - without rebuilding a maze of queues, cron jobs, and manual cleanup. Cornelia Davis breaks down how durable execution works in practice: writing “normal” code while the runtime provides durable retries, state management, and the ability to pause work, wait for a human or external change (like a quota increase), and resume right where things left off. The conversation connects these ideas to platform engineering realities - Terraform workflows, long provisioning times, and “orphan” resources - and explains how Temporal workflows and activities help teams model failure handling as a first-class part of the system. You’ll also hear why this approach is showing up in AI engineering: long-running agent workflows, frequent rate limiting, and the need to avoid re-running expensive LLM calls when something breaks near the end.

What happens when a non-deterministic AI system is asked to touch production telemetry or generate changes for an SRE pipeline? The cost of being “close enough” can be lost data, downtime, or a security incident. Cribl’s Nikhil Mungel joins Cory to break down what it takes to build AI that sysadmins can actually trust. The conversation digs into harness engineering and the practical guardrails that turn probabilistic models into repeatable, verifiable outcomes. They cover why breaking work into small chunks matters, how validation and testing become the real leverage point for AI-native development, and what “code factories” mean for review, CI, and platform reliability when teams can generate a thousand PRs an hour. Platform engineers will also hear a pragmatic take on the future of the job. The focus shifts away from typing code and toward building systems for verification, simulation, and safe deployment at scale, plus clearer ways to decide what needs human scrutiny and what can ship automatically.