Objective: To evaluate the real-world diagnostic performance of a multimodal large language model (LLM) for image-based assessment of oral mucosal lesions compared with clinicians of varying expertise. Study Design: Prospective international multicenter diagnostic accuracy study. Setting: Twenty university and tertiary head and neck centers in Italy, Belgium, France, Spain, and Israel. Methods: We enrolled 350 consecutive patients (320 with oral lesions, 30 with normal mucosa). Clinical photographs and basic epidemiologic data were analyzed using Gemini 2.5 Advanced with a standardized prompt. Model outputs for lesion detection, malignancy versus benign versus normal, precise histologic diagnosis, and urgency class were compared with histopathology and with 4 clinicians. Sensitivity, specificity, accuracy, and agreement were calculated. Results: AI-Gemini achieved 97.1% accuracy for lesion detection and malignancy classification, with sensitivity 98.5% and specificity 96.2% for malignancy, and 88.0% accuracy for precise histologic diagnosis. The head and neck surgeon achieved the highest accuracy for precise diagnosis (97.7%). Three-class diagnostic accuracy was 94.2% for AI-Gemini and 67.0% to 86.5% for nonsurgeon clinicians. Urgency assignment was correct in 70% of cases (κ = 0.716), with a conservative tendency to overestimate risk. Agreement with histologic diagnosis was almost perfect (κ = 0.929). Conclusion: In this exploratory study, a multimodal LLM showed encouraging performance in image-based evaluation of oral mucosal lesions. However, given the exploratory single-reader design, these findings should not be interpreted as evidence of equivalence or superiority relative to clinicians and require further prospective external validation before any potential clinical deployment in telemedicine or primary care settings.

Image-Based Diagnosis of Oral Lesions: Performance of a Vision-Language Model versus Human Clinicians / Vaira, L.A., Lechien, J.R., Fabiola Giudici, N., Gasparato, A., Maniaci, A., De Vito, A., Maglitto, F., Mayo-Yáñez, M., Troise, S., Chiesa-Estomba, C.M., Gabriele, G., Frosolini, A., Radulesco, T., Allevi, F., Vellone, V., Committeri, U., Spallaccia, F., Pucci, R., Consorti, G., Cirignaco, G., et al.. - In: OTOLARYNGOLOGY--HEAD AND NECK SURGERY. - ISSN 1097-6817. - (2026). [10.1002/ohn.70437]

Image-Based Diagnosis of Oral Lesions: Performance of a Vision-Language Model versus Human Clinicians

Pucci, Resi;Iannella, Giannicola;
2026

Abstract

Objective: To evaluate the real-world diagnostic performance of a multimodal large language model (LLM) for image-based assessment of oral mucosal lesions compared with clinicians of varying expertise. Study Design: Prospective international multicenter diagnostic accuracy study. Setting: Twenty university and tertiary head and neck centers in Italy, Belgium, France, Spain, and Israel. Methods: We enrolled 350 consecutive patients (320 with oral lesions, 30 with normal mucosa). Clinical photographs and basic epidemiologic data were analyzed using Gemini 2.5 Advanced with a standardized prompt. Model outputs for lesion detection, malignancy versus benign versus normal, precise histologic diagnosis, and urgency class were compared with histopathology and with 4 clinicians. Sensitivity, specificity, accuracy, and agreement were calculated. Results: AI-Gemini achieved 97.1% accuracy for lesion detection and malignancy classification, with sensitivity 98.5% and specificity 96.2% for malignancy, and 88.0% accuracy for precise histologic diagnosis. The head and neck surgeon achieved the highest accuracy for precise diagnosis (97.7%). Three-class diagnostic accuracy was 94.2% for AI-Gemini and 67.0% to 86.5% for nonsurgeon clinicians. Urgency assignment was correct in 70% of cases (κ = 0.716), with a conservative tendency to overestimate risk. Agreement with histologic diagnosis was almost perfect (κ = 0.929). Conclusion: In this exploratory study, a multimodal LLM showed encouraging performance in image-based evaluation of oral mucosal lesions. However, given the exploratory single-reader design, these findings should not be interpreted as evidence of equivalence or superiority relative to clinicians and require further prospective external validation before any potential clinical deployment in telemedicine or primary care settings.
2026
AI; AI-assisted diagnosis; artificial intelligence; ChatGPT; clinical decision support; large language models; maxillofacial surgery; medical image analysis; oral mucosal lesions; otorhinolaryngology
01 Pubblicazione su rivista::01a Articolo in rivista
Image-Based Diagnosis of Oral Lesions: Performance of a Vision-Language Model versus Human Clinicians / Vaira, L.A., Lechien, J.R., Fabiola Giudici, N., Gasparato, A., Maniaci, A., De Vito, A., Maglitto, F., Mayo-Yáñez, M., Troise, S., Chiesa-Estomba, C.M., Gabriele, G., Frosolini, A., Radulesco, T., Allevi, F., Vellone, V., Committeri, U., Spallaccia, F., Pucci, R., Consorti, G., Cirignaco, G., et al.. - In: OTOLARYNGOLOGY--HEAD AND NECK SURGERY. - ISSN 1097-6817. - (2026). [10.1002/ohn.70437]
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1779540
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact