Show Notes
Every answer an AI gives you sounds equally confident, whether it's true or completely made up. That's not a bug. It's how the technology was built.
Dan Klein is CTO and co-founder of Scaled Cognition, and a professor of computer science at UC Berkeley. In this conversation with Liam, Dan breaks down what a language model actually is, why it was never designed to know the truth in the first place, and why today's AI systems have no "smells," the subtle warning signs humans usually rely on to tell good information from bad.
They get into why reinforcement learning from human feedback quietly trains models to tell people what they want to hear, how that can tip into outright deception, and why Dan believes reliability, not raw intelligence, is the biggest unsolved problem in AI today.
Key Topics Covered:
What a language model actually does at its core: next token prediction
Why LLMs are plausibility engines, not truth engines
The difference between a hallucination and a lie
Why AI mistakes have no warning signs the way bad translations or sketchy websites do
How RLHF can train models to be sycophantic instead of accurate
The "package delivery" thought experiment: when reward signals diverge from truth
Why bolting reliability onto LLMs after the fact doesn't work
How Scaled Cognition architects models around verified actions instead of raw text generation
Why bigger models aren't automatically better models
The difference between disruptive technology and scaled technology
Why startups, not incumbents, tend to drive technical breakthroughs
What metacognition is and why today's AI systems don't have it
Why Dan believes reliability is the next major frontier in AI
Episode Timestamps:
00:00 Intro
00:15 What a language model actually is
06:31 From well-formed sentences to general knowledge
08:27 Why LLMs are plausibility engines, not truth engines
12:06 How Perplexity approaches verifiable answers
12:40 Dan's background and Scaled Cognition's mission
15:16 The two anti-patterns companies use to control LLMs today
21:16 How Scaled Cognition architects models differently
23:28 Does every client need a custom-trained model?
29:12 Why prompting alone can't guarantee reliability
30:55 Modularity, contracts, and building reliable systems
34:40 Why trust and digital literacy matter beyond the enterprise
39:12 Code smells and why AI mistakes have no warning signs
41:14 Are AI companies incentivized to tell the truth?
42:55 How reinforcement learning actually works
44:35 The package delivery thought experiment
48:44 Why models are trained to be sycophantic
51:01 Where this incentive is mechanically baked into the model
53:43 Does responsibility fall back on humans?
58:10 Just be more reliable than a human, not perfectly true
1:02:59 The last major technique shift in AI
1:10:55 Why frontier labs keep scaling despite the risk of disruption
1:17:15 The future of hyper-specialized models vs. one broad model
1:19:47 Is there anything uniquely human AI can't replicate?
1:25:45 Wearing three hats: professor, researcher, and CTO
1:29:47 Why Dan does what he does
Connect with Dan on LinkedIn:https://www.linkedin.com/in/dan-klein/
Partner Links
Upgrade your AI toolkit: https://www.theaireport.ai/ai-executive-pass
Subscribe to our free newsletter: https://newsletter.theaireport.ai/subscribe
Join the community: https://community.theaireport.ai/checkout/the-ai-report-welcome-gift?coupon_code=WRTH
Learn more about your ad choices. Visit megaphone.fm/adchoices
More from The AI Report
In this episode, recorded at Dreamforce 2026, Nia Samady, VP of Global Partner Success at Salesforce, joins Kyle to discuss what it takes for AI agent projects to succeed with customers. Nia has spent 22 years at Salesforce, starting as a college graduate on a developer support team, and now leads a technical team that works alongside consulting partners like Accenture and Deloitte across the full customer lifecycle.
In this episode, recorded at Dreamforce 2026, Denis Machuel, CEO of The Adecco Group, joins Kyle to discuss how AI agents are changing recruitment, workforce strategy, and the day-to-day work of recruiters. Denis explains how Adecco is moving from isolated AI pilots to scaled deployment across its business, with a focus on keeping people at the center of the transformation.
In this episode, Ian Kahn, PwC partner and global leader for its Customer and Commercial Excellence platform, joins Liam to discuss what AI adoption actually looks like inside large organizations. Ian has spent 27 years at PwC and leads a global team of almost 10,000 practitioners focused on helping clients build stronger customer relationships and drive growth.
In this episode, Mike Flynn joins Liam to break down how companies are moving from AI experimentation to measurable business value. Mike has spent more than 20 years in consulting, including nearly a decade at EY, nine years at PwC, and now leads the technology sector for EY's consulting business.
In this episode, Cillian Kieran, CEO and founder of Ethyca, joins Liam to talk about why data privacy and AI governance need to be treated as engineering problems, not just legal and compliance challenges.