Icon Legend

This session is not in your schedule.

This session is in your schedule. Click again to remove it.

Favorite

Best of Scientific Oral Presentations

Comprehensive Otolaryngology

Session: Best of Scientific Orals 2

Accuracy of ChatGPT-4o in Supervising Residents in Otolaryngology-Head and Neck Surgery: A Randomized Controlled Trial

Sunday, October 12, 2025

3:00 PM - 3:06 PM EDT

Location: 120

Introduction:

Objective: An increasing number of young practitioners used large language models (LLMs) in their practice. This randomized controlled trial evaluated diagnostic accuracy between GPT-4-guided postgraduate year-1 residents (PGY1) and senior otolaryngologists, addressing the increasing integration of LLMs in clinical practice.
Study design: Randomized Controlled Trial.
Setting: University Study.

Methods: PGY1 residents conducted initial consultations for 80 consecutive outpatients, with cases randomly assigned to supervision by either GPT-4o or a senior otolaryngologist. Medical history, clinical presentations, and examination findings were evaluated for diagnostic and therapeutic recommendations by both supervisors. Two blinded board-certified otolaryngologists independently assessed all recommendations using the Artificial Intelligence Performance Instrument (AIPI). The intraclass correlation coefficient (ICC) was used to measure the interrater agreement.

Results: Patient demographics were comparable between GPT-4o (n=40; 70% female; mean age 53.2 years) and otolaryngologist groups (n=40; 67.5% female; mean age 52.5 years). PGY1 diagnostic accuracy reported similar error rates (n=7) in both groups. Blinded assessment by two independent otolaryngologists revealed comparable performance between the otolaryngologist and GPT-4 for history, symptoms, findings, primary diagnosis, and management plans. GPT-4 achieved superior AIPI scores in differential diagnosis, additional examinations, and treatment explanations. The mean AIPI score for GPT-4 was significantly higher than the otolaryngologist's (14.20 ± 2.24 versus 12.34 ± 1.91; p=0.001). Interrater reliability was high (ICC=0.981).

Conclusions: GPT-4o reported equivalent or superior PGY1 supervision compared to senior otolaryngologists, suggesting potential value in clinical training settings.

Presenting Author(s)

LS

Lise Sogalow, MD

PhD student
University of Mons
Jurbise, Hainaut, Belgium

Disclosure(s):
faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.
MM

Meryem Miri, MD

PhD candidate
UMONS
Mons, Hainaut, Belgium

Disclosure(s):
faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Co-Author(s)

AM

Antonino Maniaci, MD

MD, PhD, Assistant Professor
Enna Kore
Enna, Sicilia, Italy

Disclosure information not submitted.
CC

Carlos M Chiesa-Estomba, PhD

San Sebastian, United States

Disclosure information not submitted.
SS

Sarah Saxena, MD

UMONS, Belgium

Disclosure information not submitted.
SH

Stephane Hans, MD, PhD

Professor Head of Department of Otolaryngology-Head Neck Surgery
Foch Hospital UVSQ, Paris Saclay University
Paris, Ile-de-France, France

Disclosure information not submitted.

Senior Author(s)

GB

Giovanni Briganti, MD

UMONS, Belgium

Disclosure information not submitted.

Presenting Author(s)

JL

Jerome R. Lechien, MD, PhD, FACS

Chair
University of Mons
Sombreffe, Hainaut, Belgium

Disclosure(s):
faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.