Screaming in the Cloud

AI's Security Crisis: Why Your Assistant Might Betray You

Screaming in the Cloud

Show Notes

On this episode of Screaming in the Cloud, Corey Quinn talks with Simon Willison, founder of Datasette and creator of LLM CLI about AI’s realities versus the hype. They dive into Simon’s “lethal trifecta” of AI security risks, his prediction of a major breach within six months, and real-world use cases of his open source tools, from investigative journalism to OSINT sleuthing. Simon shares grounded insights on coding with AI, the real environmental impact, AGI skepticism, and why human expertise still matters. A candid, hype-free take from someone who truly knows the space.

Highlights:

00:00 Introduction and Security Concerns 02:32 Conversations and Kindness 04:56 Niche Museums and Collecting 06:52 Blogging as a Superpower 08:01 Challenges of Writing and AI 15:08 Unique Use Cases of Dataset 19:33 The Evolution of Open Source 21:09 Security Vulnerabilities in AI 32:18 Future of AI and AGI Concerns 37:10 Learning Programming with AI 39:12 Vibe Coding and Its Risks 41:49 Environmental Impact of AI 46:34 AI in Legal and Creative Fields 54:20 Voice AI and Ethical Concerns 01:00:07 Monetizing Content Creatively Links: Simon Willison’s Blog Datasette Project LLM command-line tool and Python library Niche Museums GitHub MCP prompt injection example Highlights from the Claude 4 system prompt AI energy usage tag AI assisted search-based research actually works now POSSE: Publish on your own site, syndicate elsewhere Bellingcat Lawyer cites fake cases invented by ChatGPT, judge is not amused (May 2023) AI hallucination cases database Sponsor Simon to get his monthly summary newsletter https://simonwillison.net/ https://www.linkedin.com/in/simonwillison https://datasette.io/

Sponsor Augment Code: https://www.augmentcode.com/

More from Screaming in the Cloud

Oct 6, 2026

Corey Quinn sits down with Pasquale DeMaio, VP of Amazon Connect Customer , to talk about how Agentic AI is reshaping customer service, and why replacing humans isn’t necessarily the goal. They dig into the evolution of Amazon Connect Customer from a cloud-based contact center into an AI-powered customer experience platform, where agentic AI, real-time customer context, voice technology, and deterministic workflows work together to actually solve problems.

Sep 24, 2026

What happens when open source, AI, and decades of database technology collide?

Sep 10, 2026

AWS VP of Global Networking Matt Rehder joins Corey Quinn to pull back the curtain on the massive network infrastructure behind AWS. They explore resiliency at scale, AWS’s move toward flatter networks, the advantages of building custom hardware, and why AI is making networking exciting again. Show Highlights: (01:08) Meet AWS Networking Lead (02:01) Why AWS Avoids Global Outages (05:23) RNG Flat Network Explained (10:08) Overbuild Capacity And Custom Hardware (17:37) VPC Virtual Network Origins (19:50) Why TCP Still Wins (20:38) SRD Inside AWS (21:54) Opt In SRD Transport (25:31) Networks As Utilities (27:24) Learning And Growing Engineers (29:22) Training Talent In House (31:27) AI Rekindles Networking (34:26) Where To Learn Networking

Aug 27, 2026

On this episode of Screaming in the Cloud, Corey Quinn sits down with Adam Larsen, Principal Developer at Aurora Educational Technologies, to explore what technology and AI should actually look like in K-12 education. They discuss using tech to empower teachers rather than replace them, the risks of outsourcing critical thinking to AI, student data privacy, and what happens when schools become dependent on tools that may not be affordable forever. Adam also shares how AWS became his go-to platform for solving complex education data problems, and why sometimes the best technology is simply the one that reliably gets the job done.

Jun 25, 2026

AI-generated code is no longer just producing low-quality pull requests. According to AWS Principal Engineer and Valkey core maintainer Madelyn Olson, the quality of AI-assisted contributions has improved dramatically in just the last few months. In this episode of Screaming in the Cloud, Corey Quinn and Madelyn discuss how AI is changing open source development, the growing burden on maintainers, and how projects like Valkey are using AI to find bugs, improve security, and harden production systems. They also explore Valkey's continued growth, the future of software development, and why experience, operational knowledge, and community still matter in an age where code is becoming cheaper to create. *Since this episode was recorded without video, we thought it was the perfect opportunity to get creative with AI!

See all episodes →