Show Notes
R. Tyler Croy, a principal engineer at Scribd, joins Corey Quinn to explain what happens when simple tasks cost $100,000. Checking if files are damaged? $100K. Using newer S3 tools? Way too expensive. Normal solutions don't work anymore. Tyler shares how with this much data, you can't just throw money at the problem, but rather you have to engineer your way out. About R. Tyler: R. Tyler Croy leads infrastructure architecture at Scribd and has been an open source developer for over 14 years. His work spans the FreeBSD, Python, Ruby, Puppet, Jenkins, and Delta Lake communities. Under his leadership, Scribd’s Infrastructure Engineering team built Delta Lake for Rust to support a wide variety of high performance data processing systems. That experience led to Tyler developing the next big iteration of storage architecture to power large-scale fulltext compute challenges facing the organization. Show Highlights: 01:48 Scribd's 18-Year History 04:00 One Document Becomes Billions of Files 05:47 When Normal Physics Stop Working 08:02 Why S3 Metadata Costs Too Much 10:50 How AI Made Old Documents Valuable 13:30 From 100 Billion to 100 Million Objects 15:05 The Curse of Retail Pricing 19:17 How Data Scientists Create Growth 21:18 De-Normalizing Data Problems 25:29 Evolving Old Systems 27:45 Billions Added Since Summer 29:29 Underused S3 Features 31:48 Where to Find Tyler
Links: Scribd: https://tech.scribd.com Mastodon: https://hacky.town/@rtyler GitHub: https://github.com/rtyler Sponsored by: duckbillhq.com
More from Screaming in the Cloud
Corey Quinn sits down with Pasquale DeMaio, VP of Amazon Connect Customer , to talk about how Agentic AI is reshaping customer service, and why replacing humans isn’t necessarily the goal. They dig into the evolution of Amazon Connect Customer from a cloud-based contact center into an AI-powered customer experience platform, where agentic AI, real-time customer context, voice technology, and deterministic workflows work together to actually solve problems.
What happens when open source, AI, and decades of database technology collide?
AWS VP of Global Networking Matt Rehder joins Corey Quinn to pull back the curtain on the massive network infrastructure behind AWS. They explore resiliency at scale, AWS’s move toward flatter networks, the advantages of building custom hardware, and why AI is making networking exciting again. Show Highlights: (01:08) Meet AWS Networking Lead (02:01) Why AWS Avoids Global Outages (05:23) RNG Flat Network Explained (10:08) Overbuild Capacity And Custom Hardware (17:37) VPC Virtual Network Origins (19:50) Why TCP Still Wins (20:38) SRD Inside AWS (21:54) Opt In SRD Transport (25:31) Networks As Utilities (27:24) Learning And Growing Engineers (29:22) Training Talent In House (31:27) AI Rekindles Networking (34:26) Where To Learn Networking
On this episode of Screaming in the Cloud, Corey Quinn sits down with Adam Larsen, Principal Developer at Aurora Educational Technologies, to explore what technology and AI should actually look like in K-12 education. They discuss using tech to empower teachers rather than replace them, the risks of outsourcing critical thinking to AI, student data privacy, and what happens when schools become dependent on tools that may not be affordable forever. Adam also shares how AWS became his go-to platform for solving complex education data problems, and why sometimes the best technology is simply the one that reliably gets the job done.
AI-generated code is no longer just producing low-quality pull requests. According to AWS Principal Engineer and Valkey core maintainer Madelyn Olson, the quality of AI-assisted contributions has improved dramatically in just the last few months. In this episode of Screaming in the Cloud, Corey Quinn and Madelyn discuss how AI is changing open source development, the growing burden on maintainers, and how projects like Valkey are using AI to find bugs, improve security, and harden production systems. They also explore Valkey's continued growth, the future of software development, and why experience, operational knowledge, and community still matter in an age where code is becoming cheaper to create. *Since this episode was recorded without video, we thought it was the perfect opportunity to get creative with AI!