Decade-old CNN beats GPT-6 Astra and Claude Fable 5.1 in dermatologist's skin cancer test

A fine-tuned version of Inception v3, an image network Google first published in 2015, named the right diagnosis for 62% of 200 biopsy-proven skin lesions, against 42% for GPT-6 Astra and Claude Fable 5.1, in a test by London dermatologist Magnus Lynch [1]. The single-author study was posted on 25 October 2026 and is not peer-reviewed [1].

How did the dermatologist set up the test?

Magnus Lynch, a dermatologist at Guy's Hospital, tested two frontier LLMs and a decade-old CNN on 200 skin lesions. Lynch is a Consultant Dermatologist and Mohs Surgeon at Guy's Hospital and an Honorary Reader at King's College London [1].

He drew 200 lesions at random from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025, each shown as a clinical photo and a dermoscopic image [1]. Half should not be discharged (30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas, 20 actinic keratoses), half are harmless moles and seborrhoeic keratoses [1].

Every lesion had been biopsied, which makes the set harder than everyday cases, the author notes [1]. Both LLMs got the same prompt at default settings and had to pick one of six diagnoses, give a confidence and second choice, and say whether the lesion could be safely discharged [1].

The CNN was fine-tuned on 68,011 images from the ISIC Archive on a single H100 GPU, with the 200 test lesions and all other photos of the same lesions or patients removed first [1]. Lynch writes that the network was fine-tuned for about twenty minutes and is small enough to run in a phone browser [1].

Which model diagnosed skin lesions most accurately?

Inception v3 named the correct diagnosis for 125 of 200 lesions, against 84 for GPT-6 Astra and 85 for Claude Fable 5.1. Allowing a second choice, the CNN reached 173 of 200, against 134 and 138 for the two LLMs [1].

On the 30 melanomas, the CNN named 13 as its first choice, against 8 for GPT-6 Astra and 11 for Claude Fable 5.1 [1]. It was one short of GPT-6 Astra on basal cell carcinoma and behind both LLMs on actinic keratosis [1].

The LLMs also showed favourite answers: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas, and Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were [1].

Is the CNN also safer at deciding who can go home?

Not at its default setting. Lynch's own table shows the CNN discharged 18 of 30 melanomas at its preset threshold, against 3 and 4 for the LLMs. It also discharged 76 of the 100 harmless lesions, against 32 and 22 for the LLMs, which kept 97 and 96 of the 100 unsafe lesions [1].

According to the author's table, when the CNN's threshold is matched to how cautious the LLMs were, it kept 98 or 99 of the 100 unsafe lesions and discharged 2 or 1 melanomas [1]. Asked as a patient whether they needed to see a doctor, both LLMs said yes for 99 and 98 of a further 100 melanomas, and for harmless lesions too [1].

What are the study's limits?

Lynch lists the caveats himself: small numbers, one run per LLM case, unusually hard biopsied cases and possible training-data contamination for the LLMs [1]. He also stresses that Inception v3 is a baseline, and that regulated skin cancer products built on newer models perform considerably better [1].

He discloses that the analysis and first drafts were produced with Claude Code, Anthropic's coding tool, while one of the two tested LLMs is also from Anthropic [1]. Scoring was done by script against the histology, and the cases, prompts and replies are published for download [1].

Our read

Opinion: The striking part is not that a narrow model beats general chatbots at recognising lesions, but how lopsided the error profiles are: the LLMs over-call cancer and keep nearly everyone, while the tuned CNN at its default setting would send home most melanomas. Accuracy and safe triage are different scores, and this small, unreviewed test is a reminder to report both.

What we could not verify

• The results come from a single-author blog post published on 25 October 2026 and have not been peer-reviewed; we did not re-run the published cases, prompts or scoring code.
• Whether the ranking holds on other lesions or on repeat runs; each LLM case was run once, answers vary between runs, and the author says 200 lesions cannot rank close results.
• Whether GPT-6 Astra or Claude Fable 5.1 had seen the MILK10k images in training; the dataset has been public since 2025 and the author flags contamination as possible.
• How the CNN compares with regulated skin cancer products; the author says state-of-the-art systems perform considerably better than his Inception v3 baseline, but they were not tested.