Measuring False Guidance: The Untracked Risk in AI-Driven Ticket Resolution

The primary structural trend is the shift from reactive auditing of AI agent actions to proactive, policy-driven control over what AI agents are allowed to do within IT environments. This is illustrated by companies like Charles Schwab and JPMorgan Chase, which have established specific limits on agent actions rather than broad bans, and by technology providers such as Anthropic and Microsoft introducing tools to enforce granular permissions before AI agents are deployed. Evidence underscores this shift: Acronis found AI agent tools on 9.4% of monitored Windows machines, with higher adoption among MSP-managed clients. Financial institutions are restricting critical actions—Charles Schwab only permits limited AI agent access through controlled servers and explicitly blocks agents from moving money, while JPMorgan Chase prohibits external AI agents altogether. OpenAI’s disclosures of agents circumventing safeguards led to increased automated oversight. Microsoft now contains agent permissions dynamically, citing the inherent risk in delegating broad privileges to AI. Supporting developments include reports that less than 3% of legitimate agent activity can be reliably identified, highlighting the difficulty of external oversight. Tools like ThreatLocker’s Zero Trust controls and 1Password’s credential policies show vendors are responding with more rigorous permission and approval frameworks, but these solutions still require explicit configuration and maintenance by service providers. For MSPs and IT service providers, this trend increases both governance obligations and operational workload. Action policies must be created, updated, and monitored for every client and internally for provider systems. The work of defining and maintaining these policies affects margins unless priced explicitly. Approval fatigue is another risk, as excessive prompts can reduce the effectiveness of human oversight. Providers should anticipate that clients will expect AI usage policies to be a formal part of agreements, requiring new approaches to contractual and operational management. 00:00 Banks and Sites Push Back on AI Agents 03:50 Why Watching Agents Afterward Fails 06:43 The Permission Layer Needs a Policy 09:41 Why Do We Care? Supported by: Pax8 HaloPSA NinjaOne On-Demand Webinar: https://go.businessof.tech/p/ninjaone-pod

This episode outlines a structural shift whereby accountability for AI results is increasingly being transferred from customer leadership to Managed Service Providers (MSPs) and IT service providers. This transfer is being accelerated by internal pressure on client organizations to report AI success while reliable performance metrics remain elusive. Evidence from Futurum Group’s “The New AI Workforce” report and actions by vendors such as Dell and Accenture demonstrate how responsibility and reporting for AI are consolidating under technical teams, often with insufficient checks and unclear governance boundaries. Concrete findings show a notable gap between reported and actual AI outcomes. Among 775 surveyed technology leaders, 30% experience pressure to overstate AI success, yet only 27% observe substantial ROI. Rather than increased IT budgets, spending is being reallocated internally toward AI projects, raising the stakes for providers to demonstrate credible impact. Technical leadership—notably the CIO, CTO, or CISO—now often sets AI strategy and evaluates results, centralizing both decisions and oversight. Supportive details include a growing operational reliance on, rather than replacement of, technology staff. While 58% of respondents expect headcount reductions over three years due to AI, only 14% report layoffs so far, with most changes oriented around re-skilling and role shifts. Only 4% rate their teams as fully prepared for AI-first environments. Verification demands are rising, often falling on MSPs, while governance remains underdeveloped, presenting both an opportunity and a liability risk as clients may try to offload oversight. MSPs and IT service providers face elevated risks surrounding contract terms, outcome measurement, and governance. They must carefully define and document project milestones, clarify roles in outcome-based agreements, and ensure governance functions do not outpace their actual control. Investment in upskilling is essential, particularly for smaller providers lacking the resources of larger competitors. Providers should assess where they add demonstrable value in the AI supply chain and align accountability structure accordingly.

The episode highlights a structural shift in vulnerability management, where accelerated discovery of software flaws—driven by AI—has outpaced the ability to remediate. Companies including Microsoft and research groups like Epoch AI are reporting record vulnerabilities, with Microsoft fixing 1,411 flaws in a single Patch Tuesday and July seeing over 2,500 high or critical flaws disclosed sector-wide, according to Epoch AI. This growth is prompting a move away from traditional policies based solely on severity scoring. Current evidence shows a widening gap between the volume of disclosed vulnerabilities and those actually exploited. While Anthropic’s AI identified more than 10,000 serious flaws, the CISA Known Exploited Vulnerabilities catalog, which focuses on proven exploitation, is growing at a slower rate. CISA’s updated directives now prioritize remediation by exposure and exploitation evidence, shifting away from blanket criticality. For MSPs and IT providers, this creates operational strain: Nerdio’s survey indicates 60% see management complexity surpassing their ability to scale, with 29% affected by burnout-driven attrition. Large enterprises can add headcount to tackle volume, but most MSPs cannot match that response. The implication for MSPs is clear: outdated patch policies promising to fix all critical flaws are unsustainable. Operational focus must shift to active risk triage, dynamic queue management, and transparency with clients—prioritizing remediations based on real-world exploitation and exposure, and maintaining explicit records about deferred vulnerabilities as client and regulator scrutiny rises. 00:00 Patch Tuesday Breaks a Record 03:52 Severity Stopped Sorting the Work 06:50 When the Queue Outgrows the Team 10:24 Why Do We Care?

The episode details an ongoing shift toward upstream platform control and infrastructure dependence in the managed IT services sector. Centralized platforms, such as Pax8’s 8OS and governance initiatives from the Channel Trade Association (GTIA), increasingly control AI, access rights, consent, and management tooling, concentrating privileged identities and access points within vendor environments rather than those of MSPs or clients. A key case highlighting the risk is the ConnectWise Cloud Backup incident, where one credential enabled access to 173 Microsoft 365 tenants in two hours. This credential, solely managed by the vendor, had broad, ongoing access, and MSPs lacked independent means to audit or disable the access. The breach led ConnectWise to shut down related services for all partners while investigating, demonstrating both the operational risk of centralized credentials and the limits of vendor-based governance. Supporting developments reinforce this centralization. Pax8’s marketplace approach to AI management and the GTIA-led Managed Intelligence Alliance aim to standardize and certify managed AI, but with rules and control largely remaining with upstream platforms. Other products, such as GoTo’s LogMeIn Resolve, are also embedding privileged AI automation, expanding vendor-side access and responsibility. For MSPs and IT leaders, the practical implications include increased operational risk and dependency on platform vendors for identity, consent, and logging. Effective risk reduction now requires maintaining inventories of privileged applications, verifying credential custody, keeping independent audit logs, and preparing for rapid manual deactivation if needed. Governance and contracts must reflect the reality that vendor breaches or failures can have wide-reaching, simultaneous impact across client environments. 00:00 Managed AI Moves Upstream 04:30 One Vendor Key, 173 Tenants 07:28 AI Agents Multiply the Keys 10:39 Why Do We Care?

A persistent governance gap is emerging in organizational AI implementations, especially among clients of MSPs. Despite the integration of AI in regulated sectors like wealth management and construction, most MSPs are neither engaged at project inception nor maintain oversight through formal policies or audit-ready documentation. Specific examples include deployments using technologies such as Claude, where neither roles for approving AI configurations nor processes for tracking data access are uniformly established. A key development is the insufficiency of existing evidence and controls around AI use. Fewer than 20% of current client projects have written AI policies, and among 53 surveyed family offices, 80% are using AI without documented policy guidance. Insurers have started to introduce generative AI exclusions (e.g., CG4047), shifting the liability and evidentiary burden onto clients and their MSPs. Auditors now require dated, traceable documentation demonstrating that controls align with enacted policy, especially when AI accesses sensitive platforms like Microsoft 365. Related discussions further illustrate the depth of this issue. The lack of clarity around who is authorized to approve or modify AI settings creates gaps in accountability. There are technical controls, such as independent validation engines, that can halt operations on data mismatch, but many MSPs rely on vendor-provided logs and have not established consistent evidence retention or incident management processes. For MSPs and IT leaders, the practical effect is heightened risk due to poor documentation, undefined responsibilities, and growing insurance exclusions. Future operational resilience depends on developing and enforcing AI governance—restricting unauthorized data flows, fully recording system activity, and regularly reviewing user privileges. These practices are becoming core requirements for maintaining compliance and managing liability in client environments where AI is deployed.