Show Notes
Most organizations treat a bigger context window like a cheat code: dump every document in, skip the data work, ship. Jonathan Frankle, Chief AI Scientist at Databricks, says that's still wrong.
This is Jonathan's return visit to Invisible Machines — a conversation recorded last summer, released ahead of Databricks Data + AI Summit. His first appearance (season 2) was the MosaicML-era craft conversation: lottery tickets, mixology, mini-cupcakes. This one is the enterprise engineering thread: be a scientist, curate before you scale, and treat specification (what you actually want the system to do) as the bottleneck between raw model power and useful AI.
Robb and Josh press him on the myths that still seduce enterprise teams: million-token windows as a substitute for real data work, hyperscaler résumés as a proxy for talent, and the fantasy that unlocking every PDF in the org automatically makes knowledge useful. Jonathan's answer is consistent: measure success, test your use case, climb the ladder of techniques, and accept that multimodal is where long context actually earns its keep, not as a universal bypass for curation.
Along the way: the nuclear fusion vs. power lines metaphor; why building a benchmark is a cop-out compared to describing intent; prompts as parameters; chat-only UIs vs. a generation that never wanted buttons; LLM-oriented publishing and static FAQ pages; unlocking PDF at scale when curation gets skipped; early-adopter mistakes we'll laugh at in ten years; and why separating knowledge from reasoning is the north star, even if we aren't there yet.
---------- Support our show by supporting our sponsors!
This episode is supported by OneReach.ai Forged over a decade of R&D and proven in 10,000+ deployments, OneReach.ai’s GSX is the first complete AI agent runtime environment (circa 2019) — a hardened AI agent architecture for enterprise control and scale. Backed by UC Berkeley, recognized by Gartner, and trusted across highly regulated industries, including healthcare, finance, government and telecommunications. A complete system for accelerating AI adoption — design, train, test, deploy, monitor, and orchestrate AI agents. Use any AI models Build and deploy intelligent agents fast Create guardrails for organizational alignment Enterprise-grade security and governance Get in touch: https://onereach.ai/contact/?utm_source=youtube&utm_medium=social&utm_campaign=s7e11&utm_content=1
for SoundCloud: https://onereach.ai/contact/?utm_source=soundcloud&utm_medium=social&utm_campaign=s7e11&utm_content=1 ---------- The revised and significantly updated second edition of our bestselling book about succeeding with AI agents, Age of Invisible Machines, is available everywhere: Amazon — https://bit.ly/4hwX0a5
#InvisibleMachines #Podcast #TechPodcast #AIPodcast #AI #AgenticAI #EnterpriseAI #Databricks #RAG #MachineLearning #DataEngineering #EnterpriseEngineering #AIStrategy #AIEngineering0:00 Jonathan Frankle Returns | Databricks Chief AI Scientist · Invisible Machines 1:47 We Remember the Plants | Returning Guest Jonathan Frankle 2:22 Million-Token Context Windows: Do You Still Need to Train LLMs? 3:40 Be a Scientist | Measure AI Success Before You Scale 5:54 Hyperscaler Résumés Are Not Proof of AI Expertise 10:01 Maximize Impact | MosaicML, Databricks & Enterprise AI 13:02 Lottery Ticket Hypothesis vs. Real-World AI Impact 14:12 Nuclear Fusion but No Power Lines | Jonathan Frankle 16:08 AI Specification & Evals: Why "Build a Benchmark" Is a Cop-Out 17:59 The Smoothie Problem | From Model Power to Useful AI 18:53 Prompts as Parameters | Fine-Tuning Without Model Weights 22:46 It's Computing | Specification, Testing & Agent Design 24:44 LLM SEO, PDFs & Enterprise Data for AI Ingestion 27:35 Static FAQs, Curation & LLM-Oriented Publishing 30:26 Unlocking PDFs Scales Your Mistakes | Enterprise RAG 33:25 Knowledge vs. Reasoning | Brand Control in AI Search 34:50 Thanks for Listening | Invisible Machines
More from Invisible Machines podcast by UX Magazine
For twenty-five years, Ben Reason and Lavrans Løvlie have been trying to give services the design quality people already expect from a product. Everybody knows what a designed BMW feels like. Almost nobody loves their bank, their hospital, or their mortgage. A product is the contract. A service is everything before it, during it, and after you are done, and it includes the relationships people do not aspire to: tax, a prison, a claim.
Three years after The Song of Significance, Seth Godin returns to Invisible Machines on the day his new book arrives, The Knot: Problems Can Be Solved. The compression of his body of work is now a sorting tool. Most stuckness is not a lack of information, it's an entanglement—two incompatible wants held at once. Effort doesn’t help. You stay stuck until you name the pin, pull it out, and loosen the knot.
The demo works. People start depending on it. Then one morning you have to explain yesterday's decision to someone who wasn't in the room.
Three years after his first Invisible Machines visit, Don Scheibenreif returns with a sharper read on the trough: everybody knows generative AI, almost nobody has answered what it does to the business model. He led Gartner’s Autonomous Business research, the successor to digital business, and co-authored When Machines Become Customers. The through-line is that AI is a tool, the harder question is how to create value with it.
There’s a particular kind of exhaustion that hits when you’re building with AI. You’ve got the license. You’ve got the pilot. You’ve got a chat window that generates whatever you ask for. But then comes the wall—the workflow that almost works, the dependency that loops back on itself, victories that only come when you quietly move the goalposts. In this episode, we talk about how to get around the wall.