Data Privacy (The Derby Mill Series ep 13)

The nonprofit AI-progress assessment organization, METR (Model Evaluation and Threat Research) released a graph recently that has become “an industrywide obsession,” according to the New York Times. But are there better ways to track AI progress? Here, the Derby Mill panel weighs in. METR uses a task-completion time horizon to assess the progress of AI frontier systems. The “time horizon” is the duration that it would take a human expert to complete a set of diverse software development tasks. METR’s 50% time horizon, then, is the “duration at which an agent is predicted to succeed” at a given software development task half the time. According to METR, frontier models over the last year have increased their task-completion horizon by about an order of magnitude, from roughly 1.5 hours to about 15 hours (see graph). In this episode, Derby Mill host Ajay Agrawal begins by challenging Rich Sutton, Suzanne Gildert and Niamh Gavin: Should we expect another order-of-magnitude increase over the next 12 months? The discussion quickly turns into a debate over the authenticity of METR’s measurement. Does the METR graph, and its remarkable improvement, reflect an improvement in AI model intelligence? “I think this stuff is totally bogus,” says reinforcement learning pioneer Rich Sutton. “You have to realize how easy it is to cook these numbers.” Rich’s criticism prompts Ajay to challenge the panel: Is there a way to measure AI’s progress that is better than METR’s method? Niamh discusses her role in a fascinating initiative to assess AI models by how well they win elections in simulated environments. And what merit is there in Suzanne’s suggestion, to assess progress based on aggregate numbers of job postings for software developers? FEATURING Ajay Agrawal, co-founder and partner, Intrepid Growth Partners Niamh Gavin, senior advisor, Intrepid Growth Partners, Applied AI scientist and CEO, Emergent Platforms Suzanne Gildert, CEO, Nirvanic Consciousness Technologies, quantum physicist, co-founder of Sanctuary AI and Kindred Richard Sutton, senior advisor, Intrepid Growth Partners, 2024 Turing Award recipient, pioneer of reinforcement learning and professor, University of Alberta LINKS METR’s task-completion time horizon model explained, including the various versions of the time horizon graph at 50% and 80% success rates. Scroll down for a great FAQ explaining the effort’s various points. METR CEO Beth Barnes in conversation on 80,000 hours. The NY Times article that observed how METR’s graph has become “an industrywide obsession.” Derby Mill is created by the team at Intrepid Growth Partners and produced by Ghost Bureau.

Atinary Technologies, based in Lausanne and Silicon Valley, is pioneering autonomous scientific experimentation with AI-driven robotic research labs. In this episode, our season finale, the Derby Mill panel welcomes Hermann Tribukait, co-founder and CEO of Atinary, whose vision is to use the company’s technology to exponentially accelerate the discovery of breakthrough molecules. Named after the Spanish verb “atinar” — to hit a target — Atinary’s product is SDLabs, short for “Self-Driving Labs Platform,” a code-free agentic AI solution that enables scientists to optimize their experimentation workflows without coding or machine-learning expertise. The platform has been deployed across pharma, chemicals, and materials R&D. Earlier this year, in a significant step from the digital to the physical, Atinary opened its first Self-Driving Lab in Boston, where it runs experiments autonomously 24/7. Reported efficiencies and scaling are impressive. Atinary says it can generate in a week the data that would take a grad student an entire Ph.D. For one client, Atinary’s self-driving experimentation cut the use of an expensive catalyst by 30x, reducing cost by 97% and reaction time by 50%. Is autonomous scientific discovery realistic? And where might the technology go at the limit? “One place where all this leads,” observes Sendhil, “Chemistry will get one step closer to a solved field—and when chemistry gets close to a solved field, boy does that look completely different.” PARTICIPANTS Hermann Tribukait, co-founder, CEO and chair, Atinary Technologies Inc. Ajay Agrawal, co-founder and partner, Intrepid Growth Partners Sendhil Mullainathan, senior advisor, Intrepid Growth Partners, MacArthur Genius grant recipient and professor, MIT Niamh Gavin, senior advisor, Intrepid Growth Partners, Applied AI scientist and CEO, Emergent Platforms LINKS Atinary website Loïc Roch, Atinary CTO and co-founder, explains the technology Atinary launches its first self-driving lab Subscribe to The Derby Mill Series at our Substack (main site) or on YouTube, Spotify or Apple Podcasts Derby Mill is created by the team at Intrepid Growth Partners and produced by Ghost Bureau. DISCUSSION POINTS 00:00 Cold open 00:52 Explaining Atinary 03:40 Sample objectives 05:54 DMTA and learn 07:22 Efficiencies 09:29 Scaling challenge 12:39 Why run experiments? 15:12 A solved field 16:30 Chemistry and RL 17:47 GenAI lessons 18:44 Football fields 20:22 Chemistry as biology 21:29 Formulation challenge 23:15 Combinatorial search 26:52 Programmable biology 28:57 OFAT 30:44 Lightning round 33:12 Eroom’s law DISCLAIMER The content of this podcast is for informational and educational purposes only and should not be construed as marketing, solicitation, or an offer to buy or sell any securities or investments. The opinions expressed in this video are those of the participants and do not necessarily reflect the views of Intrepid Growth Partners or its affiliates. Any discussion of specific companies, technologies, or industries is for illustrative purposes and does not constitute investment advice. Viewers are encouraged to consult with their own financial, legal, and tax advisors before making any investment decisions.

In our latest episode, Intrepid’s Derby Mill podcast features the CEO and co-founder of UK-based PhysicsX, Jacomo Corbo, for a special in-person episode filmed at our Annual General Meeting in London. The interview, conducted by host Ajay Agrawal and panel members Sendhil Mullainathan and Niamh Gavin, comes just weeks after PhysicsX announced an oversubscribed $300 million Series C investment round that valued the company at $2.4 billion, which included Intrepid participation. The PhysicsX origin story includes Corbo and his co-founder, Robin Tuluie, working on the Formula 1 racing circuit for the Renault team that won back-to-back world championships in 2005 and 2006. (Corbo took a year off from studying computer science for his Ph.D. at Harvard after the Renault team noticed his game-theory based research and asked him to apply it to F1 racing environments, eventually becoming Renault’s chief race strategist.) Corbo and Tulule started PhysicsX in 2019 and now say that their machine-learning foundation models can accelerate simulation for industrial engineering by between 10,000 to 100,000 times. That means more efficient product design cycles, Corbo says, delivered under faster timelines, which in turn translates into better products. Based in London and New York with more than 300 employees, PhysicsX counts among its clients leading organizations in aerospace & defense, automotive, semiconductors, materials, and energy & renewables On the agenda in today’s discussion: How do the PhysicsX AI-enabled emulation models differ from conventional simulation? What capabilities does better simulation unlock for today’s industrial engineers? And how will tomorrow’s products differ as a result? In our first-ever in-person episode filmed in London, the Derby Mill tackles it all. PARTICIPANTS Jacomo Corbo, co-founder and CEO, PhysicsX Ajay Agrawal, co-founder and partner, Intrepid Growth Partners Sendhil Mullainathan, senior advisor, Intrepid Growth Partners, MacArthur Genius grant recipient and professor, MIT Niamh Gavin, senior advisor, Intrepid Growth Partners, Applied AI scientist and CEO, Emergent Platforms LINKS PhysicsX website PhysicsX press release for its last investment round, in which Intrepid participated Harvard School of Engineering alumni profile on Jacomo Corbo Subscribe to The Derby Mill Series at our Substack (main site) or on YouTube, Spotify or Apple Podcasts Derby Mill is created by the team at Intrepid Growth Partners and produced by Ghost Bureau. DISCUSSION POINTS 00:00 Cold open 01:10 PhysicsX described 03:10 AI Numerical Inference 04:00 Governing Physics Equations 05:57 Engineering Intuition Limitations 08:07 Optimization Computational Barriers 10:01 Democratizing Engineering Simulation 11:05 Deep Learning Models 11:26 Empirical Data Training 12:28 Industrial Customer Segments 14:41 Approximating Ground Truth 15:55 Expanding Design Space 17:01 Dark Factory Vision 18:56 Key Breakthrough Requirements 21:05 Training Data Scarcity 22:10 Reinforcement Learning Approach 24:59 End-of-One Customization 26:27 Influencing Physical World 28:31 Scaling Pre-trained Models 29:21 Model Attention Mechanisms 30:35 High Dimensional Physics 32:41 Reducing Exploration Costs 35:12 Manufacturing Cost Curves 36:52 Sim-to-Real Performance 38:15 Improving Causal Inference 38:40 Quantum Computing Integration 40:01 Innovativeness and Novelty

UK-based Relay is a fast-growing delivery company whose mission is to lower the cost of last-mile fulfillment. CEO and co-founder Jonathan Jenssen say his team is reinventing networks for modern e-commerce. Relay covers about 23% of the UK, has a 4% share in operating postal codes and includes on its customer list such brands as Vinted, Temu, Shein, TikTok Shop, ASOS and Next. One big challenge for Relay is that the last mile amounts to ~70% of total delivery cost. Also confounding cost decreases is the fact that the parking-to-doorstep journey amounts to ~80% of last-mile time. That all means that about half of the total cost of package delivery is in the last 100 metres of the delivery. To decrease those costs, Jenssen’s team is harnessing the power of AI to make the last few hundred metres of the delivery more efficient. Gone are the days of hub and spoke routing and half-empty delivery trucks. Algorithms provide couriers with next-best-actions, in the form of turn-by-turn driving and parking directions, tips for building entry, and doorstep handover. They also use dynamic per-route pricing and live supply/demand matching, dynamic routing and an asset-light pit stop network of delivery depots that currently includes 10,000 local shops. The Derby Mill team explores where Relay’s efficiency improvements may go at the limit of AI’s capabilities. Where could robots fit in? What if Relay had access to more consumer information? Could that allow the company to actually predict customer requests? And what does all this have to do with something called “the beer game,” an MIT Sloan business case? These questions and more are answered in episode 31 of The Derby Mill Series.

How are companies using AI to improve the healthcare journey for consumers? And how will things evolve at the limit, where wearable technology integrates with electronic medical records and medical agentic AI analyzes everything on the fly to issue recommendations and improve outcomes? In other words, what are the implications of everyone becoming equipped with their own superhuman medical agent? Seen one way, a superhuman medical agent is what League is aiming to develop. Founded in 2014 in Toronto, League operates an AI-powered healthcare platform for insurance companies and other payers or providers. The platform allows users to manage health benefits as well as their own healthcare. As of early 2026, the company has more than 60 million contracted users and more than a billion consumer interactions annually. Customers include Highmark Health, Manulife, Shoppers Drug Mart, and Medibank. Last year, League gave 100 million healthcare recommendations, of which people clicked through or followed through about 65% of the time. In this episode, the Derby Mill team explores where League’s technology might go at the limit with League founder and CEO Mike Serbinis, who aims to expand the company’s capabilities to create a 24/7 infinite care platform—basically, an all-knowing doctor who knows as much as possible about you, your healthcare data, and the science of medicine. It’s a fascinating conversation that sees Sendhil observing how little the medical profession currently knows about optimizing health, or patients themselves, and Ajay challenging Rich to decide whether superhuman AI doctors would be better than having a doctor as a spouse today. (Rich says yes.) More on our guest: A serial entrepreneur, Michael Serbinis has also founded and run Kobo, the digital reading company, cloud storage pioneer, DocSpace, and Critical Path, a messaging service. Serbinis is chair of the board for the Perimeter Institute and on the board at the Vector Institute for Artificial Intelligence. He sits on Canada's AI Strategy Task Force.