Show Notes
The more AI can do, the more we need to ask what it should and shouldn’t do. In this episode, OpenAI researcher Jason Wolfe joins host Andrew Mayne to talk about the Model Spec, the public framework that defines intended model behavior. They discuss how the Model Spec works in practice, including how the chain of command handles conflicts between instructions, and how OpenAI evolves it based on feedback, real-world use, and new model capabilities.
Chapters
00:00 Introduction 01:10 What is the Model Spec? 03:55 How does the Model Spec work in practice? 06:26 Transparency: Where to read the Model Spec & give feedback 07:51 How did the Model Spec originate? 10:02 How does the spec translate into model behavior? 11:26 What is the hierarchy / chain of command? 13:35 Handling edge cases like Santa Claus 17:41 How does the Model Spec evolve over time? 19:59 What happens when models disagree with the spec? 22:05 How do smaller models follow the spec? 23:16 Is chain-of-thought useful for alignment? 24:16 Model Spec vs Anthropic’s Constitution 26:28 What surprised you most? 26:56 How do you define the scope of the spec? 27:44 What is the future of the Model Spec? 31:16 How should developers think about the spec? 34:44 Asimov’s laws vs Model Spec 37:16 Could AI write a Human Spec?
Hosted on Acast. See acast.com/privacy for more information.
More from OpenAI Podcast
Racing is a sport of tiny margins and mountains of data. OpenAI researcher Joyce Ruffell and RaceTek Systems co-founder Chase Holden are each using AI to help teams make better use of information from the track, the garage, and the people behind the wheel. They discuss OpenAI’s research collaboration with Chip Ganassi Racing and how Chase used ChatGPT and Codex to go from hosting a NASCAR podcast to building a racing intelligence company. They also examine how AI could give smaller teams an edge without replacing the human expertise at the heart of racing, and why car culture runs so deep at OpenAI.
The old tests are getting too easy. Tejal Patwardhan leads OpenAI’s frontier evals team, which is finding new ways to measure and forecast progress as models become more capable. She and host Andrew Mayne discuss why evals matter for research, how benchmarks can break or get gamed, and what models need to be judged on next.
Last month AI found something mathematicians had missed for decades. Reasoning researchers Alexander Wei, Hongxun Wu, and Lijie Chen join the podcast to discuss how a general-purpose model helped disprove an 80-year-old conjecture from famed mathematician Paul Erdős. They walk through the moment the result started looking real, what it took to verify the proof, and what’s happened since sharing the discovery with the world. They also explore what this means for the future of math and for researchers learning to work with AI.
People are generating over 1.5 billion images a week in ChatGPT. In this episode, Product lead Adele Li and researcher Kenji Hata share some of the new use cases and trends since the launch of Images 2.0. Together with host Andrew Mayne, they trace the progress from the early DALL-E days and dive into the latest capabilities, including better text rendering, photorealism, multilingual support, world knowledge, aspect ratios, and character consistency. They also explore what comes next as image generation models evolve into more capable creative assistants.
Training frontier models isn’t as simple as adding more GPUs—one small problem and the whole coordinated dance falls apart. OpenAI’s Mark Handley and Greg Steinbrecher discuss how a new supercomputer network design, used to train some of the company’s latest models, keeps the whole system moving in lockstep, even with record numbers of GPUs. They break down Multipath Reliable Connection, a new protocol OpenAI developed with AMD, Broadcom, Intel, Microsoft, and Nvidia, and why they’re making it available for the whole industry to use.