Weekly ranking, Technology
A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
47Remix
An original benchmark by a dermatologist finds a 20-minute fine-tuned Inception v3 beat two frontier LLMs (62% vs 42%), though it still sent 18 of 30 melanomas home.
What it adds
Runs a head-to-head experiment on 200 biopsy-proven lesions showing a cheap fine-tuned 2015 CNN beats frontier LLMs. It also exposes the CNN's own dangerous melanoma discharge rate and the LLMs' characteristic biases.
Claim by claim
- 12 the author's own
- 0 an AI would say
- 0 known elsewhere
- 4 common knowledge
Its strongest original claims
- On 200 biopsy-proven lesions, the CNN got the correct diagnosis for 125 (62%) versus 84 and 85 (42%) for the two LLMs.
- A small, old specialised network recognises skin lesions far better than two frontier LLMs.
- Both LLMs misidentified an example melanoma as a seborrhoeic keratosis and said the patient could be discharged, while the CNN got it right from dermoscopy.
- The CNN was Inception v3 fine-tuned on 68,011 ISIC Archive images on a single H100, taking about twenty minutes, with test lesions excluded.
- Allowing a second choice, the CNN reached 173 of 200 versus 134 and 138 for the LLMs.
Scored by Expulp Verdict for what the piece adds beyond what is already known and published. The full verdict shows every claim with its reasons. Leaderboard, what Verdict gets right and where it is weak.