#43 - SLOs: a Deeper Dive into its Mechanics

What if agentic AI makes SRE more important, not less? Bennett Gould explains why autonomous AI systems may create more demand for reliability thinking — not less. Everyone seems to think AI is coming for SRE in a hard way. You might have heard the same story: “AI will write the code.” “Agents will handle incidents.” “Copilots will generate the runbooks.” “Automation will reduce operational load.” Yes, the job question is real. If AI can write code, summarize incidents, query observability tools, generate runbooks, and operate across systems, then engineers are right to ask what happens to the work. But here’s the part that gets missed: AI does not just automate reliability work. It creates more objects and surface areas that need to be made reliable. Agentic AI is moving from demos into real workflows. These systems are no longer just answering questions. They are querying tools, pulling context, generating changes, and in some cases taking action around production environments. That makes this a Monday morning problem. Teams are already using LLMs for incidents, documentation, observability, infrastructure, and operational decision-making. Somewhere, a team is one demo away from giving an agent access to tools originally designed for humans. That is exactly why I wanted to have this conversation. Bennett Gould is currently a solution engineer at Neubird.ai. His career in SRE and SRE-adjacent work spans large enterprises, cloud, industrial technology, and startups, including AWS, IBM, Siemens, and a YC startup. I wanted to ask him a simple question: What in the agentic AI is happening to SRE? Here are 3 highlights from our talk: 1. Agentic AI increases the reliability surface area The obvious fear is that AI reduces the need for reliability engineers. Bennett’s view was more nuanced. He was clear that engineers still need to adapt. If people do not reskill, stay current, and learn how these systems are forming, there may absolutely be pressure in the job market. But he also argued that AI could create more demand for reliability skills because production complexity is increasing. More code is going into production. More AI-generated code is going into production. More systems that people do not fully understand are going into production. And now autonomous agents are starting to enter production workflows too. That means more surface area. More automation. More operational uncertainty. More ways for things to go wrong. Bennett compared this to Terraform: Infrastructure as code created enormous efficiency gains. But it also created new ways to make very big mistakes very quickly. Before Terraform, most people could not delete all their production resources with a single command. After Terraform, that became technically possible if the system was designed badly enough. Agentic AI follows a similar pattern. With great automation comes great responsibility. Agents can help engineers move faster, query tools, summarize context, and reduce toil. But they can also amplify weak engineering practices, poor boundaries, bad assumptions, and unclear operational ownership. That is not the end of reliability work. That is reliability work entering a new phase. 2. Agents can reduce toil, but context is the ceiling One of the strongest parts of the conversation was Bennett’s explanation of where agents can help in incident response. A lot of SRE work involves moving across tools. You may need to query Prometheus, Dynatrace, logs, traces, cloud consoles, ticketing systems, documentation, runbooks, dashboards, and architecture diagrams. The problem is not always that the engineer lacks judgment. Sometimes the problem is that the information is scattered across too many tools, each with its own query language and interface. Bennett gave a simple example: an engineer might be very good at PromQL and very fast when Prometheus is the source of truth. But if the same engineer has to work in a different observability platform with a different quer

What if the hardest part of reliability has nothing to do with tooling or automation? Jennifer Petoff explains why real reliability comes from the human workflows wrapped around the engineering work. Everyone seems to think AI will automate reliability away. I keep hearing the same story: “Our tooling will catch it.” “Copilots will reduce operational load.” “Automation will mitigate incidents before they happen.” But here’s a hard truth to swallow: AI only automates the mechanical parts of reliability — the machine in the machine. The hard parts haven’t changed at all. You still need teams with clarity on system boundaries.You still need consistent approaches to resolution.You still need postmortems that drive learning rather than blame. AI doesn’t fix any of that. If anything, it exposes every organizational gap we’ve been ignoring. And that’s exactly why I wanted today’s guest on. Jennifer Petoff is Director of Program  Management for Google Cloud Platform and Technical Infrastructure education. Every day, she works with SREs at Google, as well as with SREs at other companies through her public speaking and Google Cloud Customer engagements. Even if you have never touched GCP, you have still been influenced by her work at some point in your SRE career. She is co-editor of Google’s original Site Reliability Engineering book from 2016. Yeah, that one! It was my immense pleasure to have her join me to discuss the internal dynamics behind successful reliability initiatives. Here are 5 highlights from our talk: 3 issues stifling individual SREs’ work To start, I wanted to know from Jennifer the kinds of challenges she has seen individual SREs face when attempting to introduce or reinforce reliability improvements within their teams or the broader organization. She categorized these challenges into 3 main categories * Cultural issues (with a look into Westrum’s typology of organizational culture) * Insufficient buy-in from stakeholders * Inability to communicate the value of reliability work Organizations with generative cultures have 30% better organizational performance.

A new or growing SRE team. A copy of the book. A company that says it cares about reliability. What happens next? Usually… not much. In this episode, I sit down with Dave O’Connor, a 16-year Google SRE veteran, to talk about what happens when organizations cargo-cult reliability practices without understanding the context they were born in. You might know him for his self-deprecating wit and legendary USENIX blurb about being “complicit in the development of the SRE function.” This one’s a treat — less “here’s a shiny new tool” and more “here’s what reliability actually looks like when you’ve seen it all.” ✨ No vendor plugs from Dave at all, just a good old-fashioned chat about what works and what doesn’t. Here’s what we dive into: * The adoption trap: Why SRE efforts often fail before they begin—especially when new hires care more about reliability than the org ever intended. * The SRE book dilemma: Dave’s take on why following the SRE book chapter-by-chapter is a trap for most companies (and what to do instead). * The cost of “caring too much”: How engineers burn out trying to force reliability into places it was never funded to live. * You build it, you run it (but should you?): Not everyone’s cut out for incident command—and why pretending otherwise sets teams up to fail. * Buying vs. building: The real reason even conservative enterprises are turning into software shops — and the reliability nightmare that follows. We also discuss the evolving role of reliability in organizations today, from being mistaken for “just ops” to becoming a strategic investment (when done right). Dave's seen the waves come and go in SRE — and he's still optimistic. That alone is worth a listen.

I know it’s already six months into 2025, but we recorded this almost three months ago. I’ve been busy with my foray into the world of tech consulting and training —and, well, editing these podcast episodes takes time and care. This episode was prompted by the 2025 Catchpoint SRE Report, which dropped some damning but all-too-familiar findings: * 53% of orgs still define reliability as uptime only, ignoring degraded experience and hidden toil * Manual effort is creeping back in, reversing five years of automation gains * 41% of engineers feel pressure to ship fast, even when it undermines long-term stability To unpack what this actually means inside organizations, I sat down with Sebastian Vietz, Director of Reliability Engineering at Compass Digital and co-host of the Reliability Enablers podcast. Sebastian doesn’t just talk about technical fixes — he focuses on the organizational frictions that stall change, burn out engineers, and leave “reliability” as a slide deck instead of a lived practice. We dig into: * How SREs get stuck as messengers of inconvenient truths * What it really takes to move from advocacy to adoption — without turning your whole org into a cost center * Why tech is more like milk than wine (Sebastian explains) * And how SREs can strengthen—not compete with—security, risk, and compliance teams This one’s for anyone tired of reliability theatrics. No kumbaya around K8s here. Just an exploration of the messy, human work behind making systems and teams more resilient.

Most teams talk about reliability with a margin for error. “What’s our SLO? What’s our budget for failure?” But in the energy sector? There is no acceptable downtime. Not even a little. In this episode, I talk with Wade Harris, Director of FAST Engineering in Australia, who’s spent 15+ years designing and rolling out monitoring and control systems for critical energy infrastructure like power stations, solar farms, SCADA networks, you name it. What makes this episode different is that Wade isn’t a reliability engineer by title, but it’s baked into everything his team touches. And that matters more than ever as software creeps deeper into operational technology (OT), and the cloud tries to stake its claim in critical systems. We cover: * Why 100% uptime is the minimum bar, not a stretch goal * How the rise of renewables has increased system complexity — and what that means for monitoring * Why bespoke integration and SCADA spaghetti are still normal (and here to stay) * The reality of cloud risk in critical infrastructure (“the cloud is just someone else’s computer”) * What software engineers need to understand if they want their products used in serious environments This isn’t about observability dashboards or DevOps rituals. This is reliability when the lights go out and people risk getting hurt if you get it wrong. And it’s a reminder: not every system lives in a feature-driven world. Some systems just have to work. Always. No matter what.