Show Notes
In this episode of Inference Time Tactics, Cooper and Byron sit down with Charlie and Anil from Rapt AI to tackle one of the industry's most expensive problems: GPU underutilization. With half a trillion dollars invested in GPU infrastructure running at just 20-30% utilization, Rapt AI is building AI-powered orchestration that automatically analyzes workloads and matches them to the right compute resources—no guesswork required.
We talked about:
Why half a trillion dollars in GPU infrastructure runs at only 20-30% utilization—and how a 5% drop costs $200,000 per $2M investment. How Rapt AI's platform continuously analyzes workloads and auto-optimizes GPU allocation, letting customers run 4-14 models per GPU. Real results: moving workloads from H100s to A100s at 40% of the cost, and reducing GPU footprints from 184 to under 50 while improving performance. Why 2026 becomes the year of inference as agentic workloads create unprecedented infrastructure chaos. The shift from supply problems to optimization problems—and why abstraction layers matter across multi-vendor environments. Power as the next crisis: tokens-per-watt emerging as the critical metric alongside tokens-per-dollar. How intelligent orchestration frees up data scientists and ML ops teams from infrastructure tuning to focus on AI innovation.
Connect with Rapt AI:Website: https://www.rapt.ai/ LinkedIn (Anil Ravindranath): https://www.linkedin.com/in/anilravindranath LinkedIn (Charlie Leeming): https://www.linkedin.com/in/charlieleeming/
Connect with Neurometric:Website: https://www.neurometric.ai/ Substack: https://neurometric.substack.com/ X: https://x.com/neurometricai/ Bluesky: https://bsky.app/profile/neurometric.bsky.social
Hosts: Calvin Cooper https://x.com/cooper_nyc_ https://www.linkedin.com/in/coopernyc
Byron Galbraith https://x.com/bgalbraith https://www.linkedin.com/in/byrongalbraith
More from Token Engineering
In this episode of Token Engineering, Cooper sits down with Yash, Head of AI Research at Neurometric AI, to break down TMax from AI2 (Allen Institute for AI)—a fully open dataset and training recipe that pushes small open-weight models like Qwen to near-frontier performance on terminal agent tasks. The paper argues that diversity and difficulty matter more than where the data comes from—even when that data is entirely invented by Gemini rather than pulled from real-world code.
In this episode, Cooper sits down with Neurometric CEO, Rob May, to talk through a big-news month: the show's rebrand from Inference Time Tactics to Token Engineering, the launch of tokenminning.com as a response to the "tokenmaxxing" trend, Neurometric's new token engineering platform and fundraising announcement. But the bigger conversation is about why Rob thinks "token engineering" is about to become a discipline every AI company needs—the same way DevOps or UX became non-negotiable roles—and why he's betting the company on getting there first.
In this episode of Inference Time Tactics, Cooper sits down with Yash Sharma, Head of AI Research at Neurometric AI, to break down the Pioneer Agent paper from Fastino Labs—a system that uses Claude Sonnet as an ML engineer in a box to build and improve small language models end-to-end. From cold start data curation to production failure diagnosis, the paper argues the real bottleneck in SLM creation isn't training—it's everything around it.
In this episode of Inference Time Tactics, Cooper sits down with Fletcher Wimbush, CEO of Discovered AI, to get into what’s actually broken about hiring—and how 13 years of bootstrapped recruiting, 10,000 interviews, and four peer-reviewed journal articles became the foundation for a platform now outperforming its biggest competitors. With hundreds of applicants flooding every open role and general AI tools hitting the ceiling at 70-80% resume screening accuracy, Discovered is combining behavioral science with fine-tuned small language models to get hiring right the first time.
Every day billions of voice conversations happen across games, customer service calls, and financial transactions. Almost none of them are understood by machines. In this episode of Inference Time Tactics, Calvin Cooper and Yash Sharma sit down with Carter Huffman, CTO and co founder of Modulate, to explore the AI systems that can finally understand voice conversations in real time.