Show Notes
Everyone is building AI agents, but what is inside one? This episode of Learn AI in Bits takes an agent apart and walks through the parts it is built from, so "agent" stops sounding like a single piece of software and starts looking like what it is: an architecture.
At the center is the language model, which reads the task, decides what it needs, and picks the next action. Around it sit the instructions that define the agent's job and boundaries, and the tools, ordinary software the model can call to search a database, read a file, hit an API, send an email, or run code. The model does not act directly; the surrounding program exposes each tool through a defined interface, and the model has to know what a tool does, what arguments it takes, and what it returns.
The piece that separates an agent from a chatbot is the loop. Using a coding-agent example, the episode shows the cycle in action: the model reads a failing test, calls a file-reading tool, runs the test, gets an error back, edits code, and runs it again, over and over until the task is done, a limit is hit, or a human steps in. Around that loop sit state, context management, permissions, guardrails, and observability, the pieces that carry information, restrict what the agent can do, add safety checks, and record what happened.
A key idea is the division of responsibility: the model chooses actions, and the harness around it executes them and returns the results. The episode connects this to current platforms, including OpenAI's Agents API and Agents SDK and Anthropic's description of an LLM enhanced with tools, retrieval, and memory operating through a loop. It closes on two practical points: the engineering work in agents is in the decisions around the loop, such as which tools and permissions to grant and when a human should approve an action, and you do not always need an agent, because a fixed workflow that follows a predefined path is often easier to test and cheaper to run than one that decides for itself.
Sources & References OpenAI: Introducing the Agents API — https://openai.com/index/introducing-the-agents-api/ OpenAI: The next evolution of the Agents SDK — https://openai.com/index/the-next-evolution-of-the-agents-sdk/ OpenAI: A practical guide to building agents — https://openai.com/business/guides-and-resources/a-practical-guide-to-building-ai-agents/ Anthropic: Building Effective AI Agents — https://www.anthropic.com/engineering/building-effective-agents Anthropic: Writing effective tools for AI agents — https://www.anthropic.com/engineering/writing-tools-for-agents Anthropic: Demystifying evals for AI agents — https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
Voice narration is AI-generated.
More from Learn AI in Bits
Recursive self-improvement is the idea that an AI can improve the very process it uses to improve itself, and this episode separates what's been demonstrated so far from the dramatic predictions often attached to the term. It draws a clear line between self-correction (revising one answer), automated optimization (keeping the best of several tested variants), and true recursive improvement, where the system changes the process that generates future improvements, so the next round starts with a better improver.
This week's AI News Sunday Wrap-Up covers a week where the industry moved on several fronts at once: new flagship and open models, enterprise agent governance, a serious safety warning, legal pressure, and the physical infrastructure limits behind the AI race.
Meta's Muse story is moving fast, and this episode takes a closer look at Muse Spark, the multimodal reasoning model family behind Meta's push into AI agents, including yesterday's release of an open-weight version for developers.
When a large language model generates text one token at a time, even a fast GPU hits a sequential bottleneck. This episode explains speculative decoding, an inference technique that prepares several tokens at once and checks them together, and when the extra computation pays off.
When an AI model answers slowly, the problem usually isn't its intelligence — it's how the serving system moves data through a GPU. This episode goes one level below the basic inference loop covered in an earlier episode, into GPU memory, the key-value cache, attention kernels, and the scheduling techniques used by systems like vLLM and NVIDIA TensorRT-LLM.