Show Notes
Distillation has been in the AI headlines this week, and not for the usual reasons. This episode of Learn AI in Bits explains what AI model distillation is, why it is a legitimate and widely used training method, and how it landed at the center of a security dispute between American and Chinese AI companies.
Distillation uses a powerful model as a teacher: you feed it large numbers of questions, save its answers, and train a smaller student model to reproduce the useful behavior. The student does not become an exact copy; it learns patterns from the teacher's responses. Done openly, the technique makes AI cheaper to run, faster, and easier to deploy, because a smaller model can pick up a good deal of a stronger model's behavior without the full cost of training a frontier model from scratch.
The episode then walks through the September 2026 dispute. On September 8, the NSA, FBI, and CISA warned that China-based AI companies were running industrial-scale distillation campaigns against U.S. AI companies to extract restricted capabilities from frontier models. Anthropic released its own threat report, saying it identified seven China-based labs conducting illicit distillation campaigns against Claude using fraudulent accounts, proxy services, stolen credentials, and automated requests. Among the figures Anthropic reports: an Alibaba operation that peaked near three million exchanges a day and generated more than 151 million exchanges between May and July, DeepSeek generating more than 12 million exchanges over 14 days, and Moonshot AI more than 23 million over the period studied. These numbers are presented as Anthropic's own findings and allegations, which have not been independently verified, and China has rejected the accusations and described distillation as a neutral, widely used technique.
Listeners will come away understanding the difference between ordinary knowledge distillation and the alleged unauthorized version, why the capabilities being copied (coding, reasoning, tool use, data analysis, agentic tasks, and reasoning traces) change the economics of building AI, and why a distilled model can inherit a teacher's capabilities without inheriting its safety controls. The episode closes on what Anthropic says it is doing to defend against extraction, and on why the value of a frontier model increasingly lies in the training data its own responses represent.
Sources & References NSA and Others Warn China-Based AI Companies are Distilling U.S. Frontier AI Models — https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/4592113/nsa-and-others-warn-china-based-ai-companies-are-distilling-us-frontier-ai-mode/ Anthropic: Detecting and countering misuse of AI, September 2026 — https://www.anthropic.com/threat-intelligence-report-september-2026 Reuters: US accuses Chinese AI firms of 'malicious' copying of AI technology — https://www.reuters.com/technology/us-accuses-chinese-ai-firms-industrial-scale-theft-ai-technology-2026-09-08/ Reuters: China state newspaper blasts Anthropic's calls to slow AI as 'Cold War' tactic — https://www.reuters.com/world/china/china-state-newspaper-blasts-anthropics-calls-slow-ai-cold-war-tactic-2026-09-14/
Voice narration is AI-generated.
More from Learn AI in Bits
A wave of cybersecurity disclosures has made this week feel like a turning point, with people calling it "Black October." This episode treats it as a cluster of high-impact incidents, not proof that one AI-driven campaign caused them all, and separates where AI was directly involved from where the failures were conventional.
Recursive self-improvement is the idea that an AI can improve the very process it uses to improve itself, and this episode separates what's been demonstrated so far from the dramatic predictions often attached to the term. It draws a clear line between self-correction (revising one answer), automated optimization (keeping the best of several tested variants), and true recursive improvement, where the system changes the process that generates future improvements, so the next round starts with a better improver.
This week's AI News Sunday Wrap-Up covers a week where the industry moved on several fronts at once: new flagship and open models, enterprise agent governance, a serious safety warning, legal pressure, and the physical infrastructure limits behind the AI race.
Meta's Muse story is moving fast, and this episode takes a closer look at Muse Spark, the multimodal reasoning model family behind Meta's push into AI agents, including yesterday's release of an open-weight version for developers.
When a large language model generates text one token at a time, even a fast GPU hits a sequential bottleneck. This episode explains speculative decoding, an inference technique that prepares several tokens at once and checks them together, and when the extra computation pays off.