Compliance Unfiltered With Adam Goslin

What Compliance Problems Arise when AI is Writing Your Policies?- Episode 231

Compliance Unfiltered With Adam Goslin

Show Notes

On this episode of Compliance Unfiltered, AI can speed up policy drafting, but it can also create hidden compliance risk when no one validates the result. Listen, as Todd Coshow and Adam Goslin discuss why AI-generated policies often fail in audits and incident reviews, and how to use AI for drafting without losing accountability. Episode Transcript: Today, Adam, we’re gonna talk about what compliance problems arise when AI is writing your policy. So if AI wrote your security policy and no one really reviewed it, whose policy is it? Adam Goslin: I love the compliance problems. That was good. Before I go there, every time that you’re like, “Hey, tell a few friends that may be interested in what we’re doing here,” I always had that thing. I don’t remember what the frick commercial it was back in the day, but it’s like, “And they too tell two friends, and they tell two friends, and so on and so on.” I forget what the hell the topic was. It was like some PSA or something. Todd Coshow: Sounds like a pyramid scheme. I don’t know. Adam Goslin: Nah. I’m gonna be forced to go figure that out now. Anyway, if you just go jam it into AI and it spits out a policy and nobody’s taken a look at it, whose policy is it? AI is becoming a good productive tool for compliance teams. It isn’t a problem to have AI go take a whack at the first draft. The problem is, and quite honestly, this is the same issue that organizations have had now for some time. For some time you’ve been able to go out to whatever, I’m just gonna make it up, like www.writemycompliancepolicyforme.com, and you too can fill in blanks and get some steaming pile of garbage as a policy. Some people literally do. I’ve been doing engagements, and we’re going through the policy for the annual review, etc., and there’s literally placeholders left in the damn policy for where things that they should have filled in, obviously. They didn’t do anything other than take a half-hearted whack at find and replace a couple of times and called it a policy. It’s really no different. When you’ve got AI blasting out the first draft and then you just go ahead and put your signature on the dotted line, it’s not terribly useful. Quite honestly, one of the biggest problems that poses when you’re trying to go through your engagement, the relationship between people going through compliance and the folks that are gonna be assessing compliance, it’s one where there is a certain amount of built-up trust that happens between those parties. There is absolutely no better way to erode any notion of trust building when you’re just serving up what usually is one of the earliest things. Normally when you go and sit down, you go through the overview of the company and what is the scope, and the assessor’s asking all sorts of questions so they can get their arms around it. But the very first thing that they’re actually looking at generally is the policies. Do you wanna start it off on completely the wrong foot? Go have AI write the initial draft and put your signature on the bottom line, or go grab a template policy and you haven’t really reviewed it. Policies are more than just well-written language. It’s literally a written commitment about how the organization is actually operating. If leadership is signing off on policies that aren’t reflective of reality, aren’t reflective of coverage for the various standards that you’re ostensibly going up against, if it isn’t reflecting how you’re actually doing what you’re doing, etc., you’re creating compliance risk in advance of your assessor walking in to bat you over the head. It’s really not a good look when you’re sitting there at your annual assessment or onsite and running square into that wall.

More from Compliance Unfiltered With Adam Goslin

Oct 16, 2026

PCI is evolving, and AI is reshaping payment security, vendor accountability, and compliance. Todd Coshow and Adam Goslin preview the PCI Europe Community Meeting in Edinburgh, exploring emerging AI risks, governance challenges, and the importance of industry collaboration. Discover why attending matters, how to maximize networking opportunities, and what to expect from this year's event. Plus, learn why stopping by the TCT booth could help make compliance suck a little less!

Oct 8, 2026

AI can automate compliance, but what happens when it automates bad compliance faster? Todd Coshow and Adam Goslin explore the dangers of scaling broken processes, weak evidence, and checkbox-driven compliance through AI. Discover why automation without proper foundations creates more problems than solutions, where AI genuinely adds value, and why human oversight remains essential. Learn how to identify automation opportunities, avoid costly mistakes, and build smarter, more effective compliance programs without sacrificing accuracy, accountability, or security.

Oct 1, 2026

Fresh from the 20th PCI North America Community Meeting, Todd Coshow and Adam Goslin break down what had the PCI compliance world talking. From AI and compliance fatigue to manual evidence collection and the annual audit scramble, they explore the challenges teams still face—and what’s changing. Hear why continuous compliance, smarter automation, multi-framework management, and responsible AI governance are becoming essential for organizations looking to reduce friction, improve accountability, and make compliance suck a whole lot less. Episode Transcript: Well, Adam, we’re officially back from the 20th PCI North America community meeting. Give me your overall impression. What was the energy like this year and what stood out to you? Adam Goslin: Well, I can tell you one thing for sure. I forget who it was. I had a conversation with somebody that was complimenting you on starting the pod with a different, “Oh, Adam’s the sparkle of your sterling silver,” or whatever you come up with. They’re like, “I don’t think we’ve ever heard Todd say the same thing twice. It’s pretty impressive.” So yeah, you were getting some shout-outs. Todd Coshow: Appreciate that. Adam Goslin: The show was a little bit smaller than I expected. Now, I know it was in Canada, so people have to actually travel internationally to be able to get… Todd Coshow: Shout out to Vancouver. Incredible city. Adam Goslin: Oh, hell yeah. Hell yeah. We got us some absolute rockstar poutine while we were up there. It was definitely a good time. Beautiful place, beautiful city. We were very, very lucky. The days of the conference were absolutely gorgeous. It was blue sky and sunny. Sometimes when you head to the northwest of the US, Seattle area, southwest Canada region, it tends to be a little wet sometimes. Todd Coshow: It can get a bit gloomy. You’re not wrong. Adam Goslin: Yeah, no, it held off. It was great. It was a little bit smaller. I’d say there were maybe two-thirds of the normal crowd. Todd Coshow: Sure. Adam Goslin: This year they didn’t have some new version of PCI or whatever. It was their 20-year anniversary, so there was a lot of hoopla around that and whatnot. I’m hoping that next year it’ll be in Vegas. One of the side notes that we got out of the conference was it sounds like, from the release timing for the next version of PCI, they were not specific if, right now we’re on 4.0.1, so are we going to 4.1? Are we going to 5? Don’t know yet. But whatever it is, I believe they’re going to have that ready to start previewing next year. So one, being in Vegas, someplace easy for everybody to get to, lots of fun stuff to do in that area, and if they’re gonna be releasing a new version of PCI, I’m really hoping that the numbers are up over where they normally are. The one person that I do wanna give a shout-out to is good old Gina at PCI. She’s always been involved in helping to coordinate and orchestrate the show. She does a fricking rockstar job. Todd Coshow: Absolutely. Adam Goslin: That lady needs a raise, for sure. She’s always very attentive and helpful and all that fun stuff, so I wanna give her a shout-out too. There was a high amount of engagement at the show. Because it was smaller, it has an ancillary impact, which is that the quality of those conversations tends to go up. Todd Coshow: Yeah, it does. Adam Goslin: So that part was cool. Vancouver was kind of like a homecoming for me because the very first show that we did as TCT where we had a booth was… Todd Coshow: Oh, cool. Adam Goslin: My wife, Kit, and I manned the booth in Vancouver in 2019. So it was kinda cool seeing it come to a full-circle moment on the 20-year anniversary, etc. A couple other people I wanted to give a shout-out to. First is Heidi and Jeff. Thank you so much for the annual gift box. Quite frankly, I started tucking into it before it even made it back home. So that was much appreciated. And a special shout-out to Paul.

Sep 24, 2026

What happens when your compliance champion suddenly disappears? Todd Coshow and Adam Goslin explore the hidden risks of building a compliance program around one go-to person. From tribal knowledge and scattered evidence to undocumented processes and failed handoffs, discover why single-person dependency can derail your next audit. Learn how clear ownership, documented procedures, distributed responsibility, and a centralized source of truth can create a resilient compliance program that survives resignations, vacations, turnover, and the unexpected. Episode Transcript: Today, Adam, we’re gonna talk about a fateful day, and that is the day that your compliance champion leaves. Every successful compliance program seems to have that one person who knows where everything is, understands every control, remembers every audit finding, and somehow keeps the entire program running and the coffee cups full. But here’s the question, Adam: If your compliance manager quits tomorrow, would most organizations’ compliance programs survive? Also, why do so many compliance programs become dependent on one single person? Adam Goslin: I just had this visual of people running down the hallway screaming. It’s cats and dogs and da, da, da, da. I think it’s a little extreme to go, would they survive? Well, yeah. There’s that expression out there, right? “Everybody’s replaceable.” That’s what they think. It doesn’t mean… Now, could they be replaced? Sure. But at what cost is really the question. Todd Coshow: Exactly. Adam Goslin: Most companies kind of have that central figure. Quite honestly, the main reason why for a lot of organizations that it ends up funneling down to this one person is that, at the end of the day, most organizations are managing their stuff manually. There’s too much going on. There’s too many control owners. There’s too many hands in the pie. There’s too many assessors. There’s too many states this stuff can be in for everybody to go sticking their fingers in the midst of all of the tracking, etc. If you’re using a manual system, then you’re almost forced into this notion. But not even almost forced. For every single company that I’ve ever seen that has either completely manual or hodgepodge manual and internal systems, etc., it ends up eventually gravitating to that single individual. The reason is simple. Organizations keep getting burned as they are trying to manage their programs. Somebody went in there to go and make mass updates to their stuff. Let’s pretend we’re using an Excel sheet. I go, “Oh, I need to mark these 10 things in this state.” Inadvertently, the person that’s going in to make their 10 things update accidentally blasts 57 items in the process and overwrites their statuses, etc. When you’ve got shared sheets, etc., it is not if you’re going to have a problem, but when or how many times you’re gonna experience a problem. It forces organizations to move into this notion where they’ve got the one ring to rule them all, type of deal. So you end up with that go-to person. The other thing is that whoever that central figure was, they’ve learned a lot, right? Year over year over year. It’s knowledge building that basically accumulates between their ears. I’ve been through so many audits, and I’ve learned what the assessors are looking for and their feedback. I’ve got boots-on-the-ground operational experience of running the program, etc. That person that’s at the center of the compliance efforts, they literally amass an unbelievable amount of data, information, experience. It isn’t experience that this person could just poof, walk away, and somebody else is just gonna inherit it. It’s not the reality. Over time, the knowledge, data, and information moves out of documented processes and things along those lines and lands up in somebody’s noggin.

Sep 10, 2026

PCI has evolved from checkbox audits toward continuous assurance, but are organizations truly keeping pace? Todd Coshow and Adam Goslin explore how AI, cloud-native payments, software supply chain risk, and operational compliance are reshaping payment security. They explain why continuous evidence beats annual screenshots, what the PCI Community Meeting in Vancouver can offer, and how teams can turn compliance into practical, measurable risk reduction. Ideal for security leaders and compliance professionals worldwide. Episode Transcript: Adam, join us at the PCI North American Community Meeting in Vancouver, British Columbia, Canada. 20 years later, I have to ask you, Adam, has PCI actually made payments more secure? Adam Goslin: Oh, I mean, there’s no doubt it had a dramatic impact. You got to remember back in the day, it was the absolute Wild West. The card brands were just getting their asses handed to them, and something needed to be done. The level of change from when this whole adventure first started to now is pretty startling. I think in many ways it was necessary. I think that the card industry as a whole forced a lot of organizations to start leveling up, start taking this stuff seriously. I believe it was astronomically helpful as we’ve headed into this arena. Certainly, we’re seeing a lot of shifts in the marketplace where we’re moving away from a checkbox approach, etc. It’s hard to remember. I was looking up, when did PCI V1 officially get released? It was middle of December in 2004. Todd Coshow: Oh, wow. Adam Goslin: It was like the first version of it, type of a deal. It was 2006 that the PCI SSC was officially formed. Then they started to make enhancements. But there have been a whole myriad of different changes and modifications to the standards over that time. The payments arena really has made a fairly startling shift from just checking the box, annual validations, into more of a continuous security or, as we here at TCT like to call, operational mode, type of a deal. The acceleration of AI, both for the white hats and black hats, has enabled attackers to move as quickly as defenders. So it remains a very, very vibrant and exciting arena. Todd Coshow: No doubt. Well, what do you think the biggest conversations at this year’s community meeting will be? Adam Goslin: Yeah, I think there’s a lot. New and exciting things happening in the payment space will certainly be one. Bar none, a big participant will be a lot of topics surrounding AI. AI, its use and benefits and all that fun stuff. Honestly, I’m kind of hoping that somebody whips a little bit of realism in there about the dangers of AI as well. I think I’ve mentioned once or 80 times about AI zombie walk. I think there’s a lot of signs in the marketplace right now that the AI zombie walk mentality isn’t necessarily the best idea in the grand scheme of things. I think there’s a lot more reasonability starting to enter into the mix. That’s what I’m hoping at least. Certainly, a lot of discussions around operational compliance, automation of evidence collection, various ways to streamline your engagements, looking at threat intelligence and attack trends as they’re going. Certainly, one of the cool parts about being at the conference is getting updates direct from the council, what’s happening, what’s coming soon to a theater near us, all of that fun stuff. We’ve got all that coming our way as well. Todd Coshow: Indeed. What’s one thing every first-time attendee should do in Vancouver? Adam Goslin: Well, number one, I’m just gonna underscore this big time.

See all episodes →