The ChatGPT Health controversy: what the viral Nature Medicine study missed with David Fraile Navarro
PULSEShow Notes
Welcome to Pulse: Amplify, where we sit down with the leaders and changemakers shaping the future of health.
A recent Nature Medicine study went viral after reporting that ChatGPT Health under-triaged more than half of emergency cases when tested using clinician-written scenarios. The finding raised serious concerns about whether consumer AI tools are safe for medical triage.
But researchers from Macquarie University’s Australian Institute of Health Innovation took a closer look at the study design and suspected the results might reflect the evaluation format rather than the AI’s clinical capability.
In this episode of Pulse Amplify, Louise and George speak with David Fraile Navarro about their follow-up study testing five frontier AI models across more than a thousand trials. Their research suggests that when AI systems are evaluated using more natural, patient-style interactions rather than exam-style prompts, triage performance improves significantly.
The discussion explores why prompt structure, forced answer formats, and restrictions on clarifying questions can dramatically alter model behaviour, and why designing realistic evaluation methods is essential as millions of people begin using AI for health advice.
The conversation also examines broader questions: How should AI triage tools be evaluated? What role should clinicians play in AI-mediated care? And what do patients need to know before trusting AI with health decisions?
References Ramaswamy A. et al. (2026). ChatGPT Health performance in a structured test of triage recommendations. Nature Medicine. https://www.nature.com/articles/s41591-026-04297-7
Fraile Navarro D, Magrabi F, Coiera E. (2026). Evaluation format, not model capability, drives triage failure in the assessment of consumer Health AI. Zenodo. https://doi.org/10.5281/zenodo.18975048
Connect with David Fraile Navarro: LinkedIn
Visit Pulse+IT.news to subscribe to breaking digital news, weekly newsletters and a rich treasure trove of archival material. People in the know, get their news from Pulse+IT – Your leading voice in digital health news.
Follow us on LinkedIn Louise | George | Pulse+IT Follow us on BlueSky Louise | George | Pulse+IT Send us your questions [email protected] Production by Octopod Productions | Ivan Juric
More from PULSE
This week on Pulse: Hot Topics, Louise and George ask
Is artificial intelligence an existential threat to humanity? Or is the greater danger what humans choose to do with it?
This week on Pulse: Hot Topics, Louise and George ask who gets to deliver healthcare, as AI agents, a dating app, Apple and online platforms all push into territory that once belonged to clinicians and health services.
This week on Pulse: Hot Topics, Louise and George break down: OpenAI's Agent Breaches a Medicare Portal – An OpenAI agent that "didn't accept no for an answer" pulled non-public files from a legacy Medicare statistics portal, turning a benign research task into a wake-up call for healthcare's ageing legacy systems. AI Scribes Meet the Law – A Washington State court ruled a patient had no right to the ambient AI recording of her own consult, while an Australian legal analysis warns scribes may concentrate liability on the doctor who signs the note. The Medicare Pipeline Nobody Designed – An Auditor-General's report shows software increasingly marketed to optimise Medicare billing with AI, while the Health Department switched off its own AI fraud-detection model because staff couldn't keep up with what it flagged. The US Prepares to Pay AI Doctors – A New York Times investigation finds US health officials fast-tracking AI that diagnoses, prescribes and treats, including reported discussions about paying AI doctors 60 to 80 per cent of what human doctors earn.
Healthcare faces a global workforce crisis - but what if we’re asking the wrong question about AI?