
Show Notes
Rob and Justin had a plan. Scale Justin's brain across the entire P3 consulting team. Build an AI agent that bottled up his frameworks, his instincts, the way he navigates AI conversations with clients. In theory, everyone gets smarter overnight. It was a solid idea. The tech worked. The knowledge base was deep. The guardrails were tight. And almost nobody used it. Not because it was broken. Because the team wasn't waking up thinking, "Man, if only I could channel Justin right now." That wasn't the fire in front of them. So instead of feeling like leverage, the agent felt like homework. And that's the punchline. You can build something powerful and still miss the mark. No one was losing sleep over not having this tool. No one's bonus depended on it. So it drifted. Not rejected. Just... optional. That's a brutal place for a "strategic initiative" to land. The fix isn't a better tool. It's sequencing. Define the services, train the team, build the human infrastructure that makes the tool land on a surface that's ready for it. Every AI project that has worked traces back to the builder being a direct stakeholder. Not adjacent to the problem. In it. Proximity to the pain is doing a lot of work that no amount of clever architecture can replace. When leaders are the ones excited about AI and employees are the ones expected to use it, you've got a stakeholder mismatch. And that mismatch is quietly killing more AI initiatives than any technical failure ever will. If you're planning a rollout, or already wondering why yours isn't sticking, this episode is for you. Be sure to subscribe on your favorite podcast platform for new content delivered directly to your inbox.
More from Raw Data with Rob Collie
We just rebuilt our entire website in five weeks. Rob figures getting the same result the old way would have cost at least a million dollars. No, AI didn't replace the marketing department. Nobody got fired. And nobody found a magical button labeled "Make Website." We just did something that would have been completely out of reach for us a year ago. Our old website had been hanging around since 2019, which is a long time for any website and an eternity for a company that's changed as much as we have. We'd looked at replacing it. We even priced out a $300,000 rebuild and decided against it. Spending that much money, investing months of our own time, and still potentially ending up with something we'd have to talk ourselves into liking? We'd already played that game. We weren't eager for a rematch. Then five weeks ago, our CTO started building. What happened next reminded Rob of something he figured out about BI more than a decade ago, except this time the implications go a whole lot further. In this solo episode of Raw Data, he walks through where the million dollar figure comes from and why the most interesting thing about our new website has very little to do with websites. If your company has been putting something off because the old way of doing it is too painful, press play.
There are two ways to build backwards. One starts with the answer you actually want and works its way toward the plumbing. The other starts with the plumbing and hopes the answer is in there somewhere. Consulting has historically been awfully fond of option two. AI makes that harder to defend. When you can get something working in front of the business early, there's no particularly good reason to disappear into the infrastructure for six months first. Give people something they can see. Something they can use. Something they can tell you is wrong. Then fix it while fixing it is still easy. Better yet, let what you learn change what you build next. Rob and Justin start with AI and eventually wander into the hazards of building in the dark, the value of building in plain sight, and a particularly useful new term: "Byzantine Consulting." It's a conversation about working backward from the answer instead of forward from the infrastructure. Because if nobody can react to what you've built yet, there's a decent chance you haven't built the important part. Also in this Episode: Rob's Substack
Rob has a lot of Claude folders now. One knows the book. Another knows the company planning. Others have picked up meeting notes, briefing docs, LinkedIn ideas, strategy work, and whatever else was useful enough to save at the time. None of this felt like a problem while it was happening. Quite the opposite. The whole setup was working well enough that Rob kept giving it more to know. Then one of those markdown files started telling him what he was and wasn't supposed to do. Rob and Justin follow that thread into a much bigger problem than one weird sentence in one weird folder. Once AI starts carrying knowledge from one place to another, remembering things, connecting things, and helping everybody work faster, a few very old problems start wandering back into the room wearing new clothes. The BI crowd may recognize them. Rob certainly did. And one of his solutions will be immediately familiar to anyone who has ever opened a Power BI model, looked at the relationships it helpfully created for them, and started deleting.
For a minute there, it looked like the AI wars would come down to who built the smartest model. Rob's not buying that anymore. Take Claude. Rob's increasingly convinced that what makes Anthropic so sticky isn't Claude itself. It's Claude Code. It's Cowork. It's the software wrapped around the model that makes the whole thing so ridiculously useful. Great news for Anthropic, except for one tiny problem: software can be copied. And when the models themselves are interchangeable enough to live in a dropdown menu, you have to start wondering what any of these companies really have that somebody else can't recreate. That question leads straight to Microsoft, which may be holding a much better hand than it gets credit for. Everyone else is trying to worm their way into your email, your files, your chats, and the rest of your working life. Microsoft is already sitting inside the castle. From there, Rob and Justin talk through the increasingly strange economics of all this, whether actual humans using your product become the moat that matters, and finally, the proposed fix for our data center problem that involves launching the data centers into space. Give it a listen for Rob's take on that one, starting with the minor inconvenience of physics.
For years, businesses learned that having data and being able to use it are two very different things. We cleaned it, structured it, modeled it, argued about which version was the truth, and built entire careers around making messy information useful. Now AI is creating the same problem all over again. Except this time, it's knowledge. In this episode, Rob explains why everything we learned from the data era is suddenly relevant again. Companies are sitting on enormous amounts of knowledge scattered across documents, conversations, systems, processes, and people's heads. AI can consume more of it than ever before, but that doesn't magically make it organized, trustworthy, or useful. Which means a lot of those supposedly "old" data skills are about to look awfully cutting edge again. Rob also introduces The Goldilocks Altitude, his new Substack for ideas that live somewhere between the 30,000 foot AI think piece and the technical weeds. This episode features one of the first: "Everything We Did for Data, We Now Need to Do for Knowledge." Listen now, then head over to The Goldilocks Altitude. If knowledge is having its data moment, there's going to be plenty to talk about. Also in this episode: The Goldilocks Altitude