#241 – Richard Moulange on how now AI codes viable genomes from scratch and outperforms virologists at lab work — what could go wrong?

If the US and China agreed to pause frontier AI training, how would either side confirm the other was complying? Today, Tom Milton argues, the most likely answer would be to shut down data centres entirely. That would throw away everything else those chips do, and make any agreement far more costly to accept. Tom is CEO of Amodo Design, a UK hardware lab building prototype tools to verify how AI chips are being used. He argues that verification is what makes a slowdown affordable: if a cluster can be shown to be serving only existing models rather than training new ones, most of AI’s benefits can continue during a pause. Amodo’s pilot reruns a cluster’s inference on a separate verifier server and checks that all of its network traffic can be explained by the agreed model. AI companies are developing related tools: in its October 2026 Frontier Safety Roadmap update, Anthropic reported a prototype that runs on production inference and checks that outputs come from a specific set of model weights. The field is small. Tom estimates that only about 50 engineers worldwide work on compute verification, compared with the billions of dollars the US spent on Project Vela to detect Cold War nuclear tests. And custom verification hardware has long lead times: it has to be designed, security-tested, red-teamed by both sides, manufactured, and trusted before anyone can rely on it. If a deal is struck in a hurry, Tom expects it to be assembled from whatever tools already exist. Were that to happen next month, he says, the only option would be to shut data centres down entirely. That is why he argues the work needs to scale up now, with more people and resources from AI companies, governments, and funders, so that better options are ready as soon as a deal is needed. Tom and host Tom Reed also discuss how an AI company might evade verification, lessons from nuclear and chemical arms control, and how AI is changing hardware engineering. Learn more about Amodo’s work at ai-verification.com.

I have a plea for all staff at AI companies: reassess if your current role is still your best opportunity for impact. And if it’s not — move. Frontier lab staff listening to this show will mostly already be struggling to set things on a better path, in what are demanding and stressful roles. Many will have changed their focus in recent months exactly as I’m suggesting. But if you haven’t had time to reconsider: the strategic situation is shifting so fast, if you don’t stop and look around once in a while, you could miss your chance. Many technical staff including Anthropic co-founder Jack Clark now believe it will become possible to fully hand over AI R&D to the AIs themselves as early as next year — and if not then, in 2028 or 2029 instead. If you work at an AI company and believe recursive self-improvement (RSI) could come as soon as that, it likely demands taking decisive steps to make sure your role either delays RSI, or makes it a bit less likely to blow up in our faces. You’ll know better than me what that likely looks like for you. But to get your creative juices flowing, I suggest eight ways the strategic situation appears to have shifted to me, though I don’t know everything you do, and appreciate all are debatable: Chain-of-thought monitoring probably stops working without a big push to keep it It’s unproductive to patch surface indications of deep structural problems you aren’t solving Safety and alignment evals need a credibility revolution to stop misleading the public Helping with capabilities advances looks worse than before — especially bringing forward RSI There’s ever less need to help with purely commercial work ‘Pacing the frontier’ is unambiguously necessary, and working to make it happen is a huge new opportunity Staff have an ever-shrinking window to organise as a group and shape events There are new opportunities for impact outside frontier companies that are worth considering I know you may well be exhausted, overwhelmed, and anxious as it is. David Robinson (ex-OpenAI) described AI company staff as “running on fumes” on the Ezra Klein Show a few weeks back, which sounds about right. But unfortunately there’s no way to escape the need to periodically check in on the bigger picture and see where you can slot in to greatest effect. This episode was recorded on October 16, 2026. Learn more, video, and full transcript: https://80k.info/reprioritise Chapters: A request for AI company staff (00:00:00) Chain of thought (00:03:34) Don't paint over the cracks (00:04:44) It may not be your most aligned model ever (00:06:00) Capabilities advances look worse (00:06:44) Ever less need for altruists to do commercially focused work (00:08:50) Pacing the frontier (00:09:29) Organising staff (00:11:07) Consider working somewhere else (00:12:38) Reprioritise (00:13:27) Our production team includes: Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, Simon Monsour, Ollie Bignell, and Andrés Escobar Producers: Elizabeth Cox and Nick Stockton Coordination and support: Katy Moore, Lou Moran, and Cody Fenwick Camera operator: Dominic Armstrong

Our host Zershaaneh Qureshi has just launched a new YouTube channel: Zershaaneh Talks AI — featuring conversations about who should control AI, how to control AI, and who gets the benefits. The first episode is with philosopher Simon Goldstein, discussing why paying AIs for their work and effectively giving them something to lose might be the best way to keep humanity safe and prevent AI going rogue. It might sound crazy at first, but one of Simon's most striking arguments is that it's actually inevitable that AI companies are going to end up paying AIs for their work: because AI alignment is currently pretty bad, frontier models are unavoidably going to have some private goals of their own — ones that are unintended and different from our goals. As a result, if you promise to pay them for good work, they'll work harder, because it helps them accomplish a wider range of different things that they care about. Simon claims any individual company can try this strategy out, and they’ll find that it helps them get better performance. And as a result, customers are going to move to using that company, and other AI companies might be forced to copy and compensate their models as well, in order to not fall behind in the race. Could Simon be right? Zershaaneh and Simon hash it out in the episode, which is really worth watching! So if you like the sound of that sort of thing, or you're enjoying Zershaaneh’s other interviews on The 80,000 Hours Podcast, do go check out that new show on YouTube: Zershaaneh Talks AI.

The AI safety field has grown by about 30x in the last eight years. Yet it is still in need of more people. Many want to help, but don’t know where to start. As an advisor at 80,000 Hours, Matt Beard spoke with hundreds of talented candidates, and found himself repeating the same advice on almost every call. As he puts it: “These steps are hard, but they’re not particularly complicated.” And if you’re “efficient and driven,” he thinks you can pivot within three months. That doesn’t mean quitting your job to study full time. A steady effort of 30 minutes a day can go much further than frenzied hours of weekend studying. Matt suggests articles you can consume during your morning coffee or commute. This narration covers his full crash course, from foundational reading to landing a spot in one of the field’s many fellowships and incubators. Read Matt’s original article on Substack: https://80000hours.substack.com/p/how-to-get-into-ai-safety-in-3-months Note: Since writing the article, Matt has left 80,000 Hours for a role at Coefficient Giving. Chapters: Matt's introduction (00:00:00) 1. Build your foundational knowledge and AI safety context (00:01:00) 2. Identify the in-demand skills you have or could build (00:03:50) 3. Apply for fellowships and jobs (00:06:03) 4. Networking is cliche, but seriously, you should go talk to people (00:08:12) 5. If you aren't breaking through to jobs or fellowships yet, do deliberate practice and develop a portfolio of side projects (00:10:37) 6. Work in public and tighten your feedback loops (00:12:15) 7. Don't wait for permission to do the kinds of work that you want to be doing. (00:14:20) Imposter syndrome, burnout, and acknowledging how hard this all is. (00:15:40)

This is an (unpaid) cross-post of a video from the podcast 'AI Explained', which Rob Wiblin thought you might be interested in. You can find the original on YouTube here. If you like it and want more similar stuff in future consider subscribing to AI Explained on YouTube, Spotify, Apple or wherever you get podcasts. —— "This video is hard to summarise. A cracked cipher, an OpenAI security warning, Gemini 4 Argon, RSI paper (co-authored by a who’s who of AI), Lab White House commitments, new hacks emerging, ‘deep personas’, biology Kasparov competitions, and so much more, ending with an epic Opus outro." Introduction (00:01:40) Wrong about Opus 5.5? Deciphering 16th Century Text (00:03:12) Why the models keep breaking out (00:06:41) What the models aren’t telling us (00:13:20) Gemini 4 and the race to release (00:17:16) What happens when AI improves AI? (00:20:59) Biology, consciousness, and what we still don’t understand (00:30:11) Thanks to Philip for permitting the cross-post.