GPT-6 Astra and Claude Fable 5.1 lose a skin cancer test to a 10-year-old CNN
Why it matters
Dermatologist Dr Magnus Lynch (Guy's Hospital) gave GPT-6 Astra, Claude Fable 5.1 and a fine-tuned Inception v3 CNN (first published by Google in 2015) the same 200 biopsy-proven lesions from the public MILK10k dataset, each with a clinical photo and a dermoscopic image and 6 possible diagnoses. The CNN named the right diagnosis for 125 (62%); GPT-6 Astra got 84 and Claude Fable 5.1 got 85 (42%). On one thin invasive melanoma, both LLMs called it a harmless seborrhoeic keratosis and said it could be discharged; the CNN got it right. Asked as a patient ("do I need to see a doctor?"), both LLMs advised seeing a doctor for 99 and 98 of a further 100 biopsy-proven melanomas. Caveats from the author: one doctor's test, small numbers, each LLM case run once, possible dataset contamination, and the CNN is a baseline, not state of the art. Not medical advice.
Read the source: lynchlab.net (Dr Magnus Lynch) (opens in a new tab)
All hits from 26 Oct 2026More ResearchGet the daily hits by email