Show Notes
This week, we're doing a deep dive into how marketing companies are poisoning AI search results by manipulating Reddit. You may remember when Google’s AI search results first launched, it recommended that people put glue on their pizza. Well that happened because it scraped a 10 year old Reddit comment. We’ve learned over the last year or so that this sort of thing can be done on purpose, and brands are taking advantage of it. There’s been the rise of AEO or GEO, which stands for AI Engine Optimization or Generative Engine Optimization. Basically this is trying to get mentions of your brand into web content that’s likely to be scraped by AI tools. It’s the new version of SEO and lots of marketers and companies are trying to do it.
The most reliable, easiest way to do this appears to be by putting brand mentions onto Reddit. Reddit’s volunteer mods have noticed an increase in bot accounts and entire sequencing efforts—where a post and its comments are all basically done as a stealth ad—intended to boost brands. I wrote an article about this a few weeks ago, about r/biohackers banning mentions of peptides, which were a popular promoted class of product. After we wrote that article, researchers from Cornell University reached out to me about a new study they had just done.
The research is called “Deep-research agents can be poisoned via user-generated content,” which provides a mechanism for the ways reddit, wikipedia, and other sites that allow users to post are being attacked by brands doing AEO: "We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently," the study says.
We spoke to two of the researchers, Hal Triedman and Tingwei Zhang, about this problem and what, if anything can be done about it.
Deep-Research Agents Can Be Poisoned via User-Generated Content: https://arxiv.org/abs/2605.24245
Youtube Version: https://youtu.be/2uG8ohZHOD8 Learn more about your ad choices. Visit megaphone.fm/adchoices
More from The 404 Media Podcast
We start this week with Jason’s article about how everyone is apparently reverse-engineering everything. After the break, Joseph tells us about a big AI shakeup at consulting giant Accenture. In the subscribers-only section, Sam and Jason discuss the Character[.]ai bankruptcy.
Big tech is rushing to build hundreds of data centers across the planet to keep up with the computing demand of artificial intelligence. The tech companies are promising the massive buildings will bring jobs and prosperity when they’re built, but communities across the country aren’t buying it and they’re fighting back. People are concerned about water, noise, pollution, and the rising cost of electricity. Anyone who already lives next to a data center knows those fears are well founded.
We start this week with Joseph’s article about Raven, previously known as Geospy. This is an AI tool that can geolocate photos, and now says it can pinpoint a specific house based just on one photo taken inside. After the break, Jason explains why a judge ruled a particular Flock search as unconstitutional. In the subscribers-only section, we talk about the wild AI video that has derailed a manslaughter case.
Sponsor: Take back control today: go to incogni.com/404media and use code 404MEDIA for 60% off an annual plan. That's incogni.com/404media, promo code 404MEDIA—risk-free with a 30-day money-back guarantee.
We start this week with a story from our old friend Lorenzo Franceschi-Bicchierai, about how cops can bypass the inactivity reboot on iPhones. After the break, Jason tells us how people are losing their minds over an LLM ‘torture’ prison. In the subscribers-only section, Sam explains what happened in a wild ChatGPT case in New Mexico.