The a16z Show

Why Real-World AI Is Harder Than the Benchmark | EliseAI’s Mario Martone

The a16z Show

Show Notes

What changes when an AI mistake doesn’t just hurt a benchmark score, but affects someone’s housing or healthcare? EliseAI Head of Applied Research Mario Martone argues that deploying AI in essential industries requires a very different standard of reliability. Mario joins MTS to explain how EliseAI handles roughly 5 million phone calls a month, why LLMs aren’t the right solution for every problem, and how the team combines language models, traditional machine learning, neural networks, and internally built evaluations to decide what actually works in production. Drawing on 15 years studying theoretical physics, Mario also explains how the scientific method shapes EliseAI’s research culture, from testing new models against its own benchmarks to fine-tuning open-source models and automating increasingly complex workflows across housing and healthcare. And he makes the case that applying AI to the real world can produce problems every bit as intellectually difficult as the ones he encountered studying string theory.

More from The a16z Show

Oct 24, 2026

What does it take to build a company when AI is changing the rules this quickly? Ben Horowitz argues that technology may make companies easier to start, but the hardest parts of entrepreneurship, making decisions under uncertainty, building momentum, and managing your own doubts, aren't going away. Speedrun's Andrew Chen sits down with Ben to discuss why AI feels different from previous technology shifts, whether billion-dollar companies can be built with tiny teams, and why Ben thinks the application itself may eventually become less important as agents work directly against data. They also get into defensibility in AI, what founders should look for in an investor, how venture capital is changing, and why the best entrepreneurs aren't defined by a lack of weaknesses, but by having a strength or insight that's genuinely exceptional.

Oct 23, 2026

Amazon expects to spend roughly $220 billion in CapEx in 2026, yet CEO Andy Jassy says the company is still leaving meaningful AI demand on the table.** He believes the buildout could ultimately help make AWS a $1 trillion annual run-rate business.

Oct 22, 2026

a16z is expanding a16z crypto into a16z Frontier, a broader mandate to back founders building at the edges of what's possible. Chris Dixon joins Marc Andreessen, Ben Horowitz, and Erik Torenberg to explain why founders should increasingly ask not “is this too ambitious?” but “is this ambitious enough?” — and how AI could pull decades of technological progress forward across hardware, robotics, entertainment, science, and entirely new forms of media.

Oct 21, 2026

AI agents can reason over enormous amounts of information, but much of the world’s actual work is still trapped in PDFs, spreadsheets, scans, and other formats that models struggle to reliably understand. Reducto co-founder and CEO Adit Abraham argues that solving this document layer is essential if agents are going to take on consequential work in healthcare, legal, finance, and insurance. Adit joins a16z General Partner Jennifer Li to explain why traditional OCR breaks down on messy real-world documents, how Reducto combines vision-language models with specialized infrastructure to reach high accuracy without sacrificing cost or latency, and why frontier models can silently drop information on long extraction tasks. Reducto now processes billions of pages each month, including its seven billionth page the morning of the recording. They also discuss Reducto’s R1 model, agents that can review and correct their own document extraction, and the company’s push beyond simply reading files toward letting agents reliably work with and edit them. Underneath it all is a higher-stakes problem: when a misread word can affect a healthcare claim or financial decision, getting the input right matters just as much as making the model smarter.

Oct 20, 2026

More than 300 million people use ChatGPT for health and wellness questions each week. OpenAI’s Karan Singhal thinks that’s only the beginning: the longer-term goal is AI that can help people navigate their health proactively, while making trustworthy medical intelligence dramatically more accessible. Karan joins a16z Partner Jay Rughani to explain why OpenAI trains healthcare expertise directly into its general-purpose frontier models, how it works with hundreds of medical experts to evaluate safety and disagreement, and why the company sees AI as a complement to doctors rather than a replacement for medical care. They also discuss integrations with Epic, AI copilots for clinicians, and what happens as personal agents begin navigating appointments, health records, and other fragmented parts of the healthcare system. Looking further ahead, Karan makes the case that the cost of medical intelligence could continue to fall dramatically, allowing AI to act as a safety net for patients and clinicians, surface insights from years of health data, and potentially accelerate the discovery and validation of new treatments.

See all episodes →