Skip to main content
AAO-HNSF 2025 Annual Meeting & OTO EXPO Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Rate ePoster
Video

Care Equity, Quality, and Safety

(A104) AI Models for Dysphonia Patient Education

Monday, October 13, 2025
12:00 PM - 1:00 PM EDT
Location: OTO EXPO, Posters

    Presenting Author(s)

  • Rebecca An Ho, BA

    Medical Student
    Rutgers New Jersey Medical School
    Haworth, New Jersey, United States

    • Co-Author(s)

  • Rebecca An Ho, BA

    Medical Student
    Rutgers New Jersey Medical School
    Haworth, New Jersey, United States

  • Esha Shah, BA

    Medical Student
    Rutgers - New Jersey Medical School
    Jersey City, New Jersey, United States

  • John Sebastian De Armas, MD

    Resident Physician
    Rutgers New Jersey Medical School
    Newark, New Jersey, United States

  • Kenneth Yan, MD, PhD

    Assistant Professor - Laryngology
    Rutgers-New Jersey Medical School
    Englewood Cliffs, New Jersey, United States

  • Rachel J. Kaye, MD

    Associate Professor
    Rutgers New Jersey Medical School
    Newark, New Jersey, United States

    • Senior Author(s)

  • Rachel J. Kaye, MD

    Associate Professor
    Rutgers New Jersey Medical School
    Newark, New Jersey, United States

Disclosure(s):
Rebecca An Ho, BA: No relevant relationships to disclose.
Esha Shah, BA: No relevant relationships to disclose.
John Sebastian De Armas, MD: No relevant relationships to disclose.
Kenneth Yan, MD, PhD: No relevant relationships to disclose.
Rachel J. Kaye, MD: No relevant relationships to disclose.


Introduction: Chat Generative Pre-Trained Transformer (ChatGPT) and Google AI Overview (GAO) have been increasingly utilized in the medical field. However, AI model responses have not been compared with standard laryngology guidelines. This study aims to evaluate and compare ChatGPT, GAO, and Dysphonia International (DI) responses to patient queries related to dysphonia.

Methods: Two board-certified otolaryngologists (KY, RK) and an otolaryngology resident (SD) each blindly rated ChatGPT, GAO, and DI’s responses to questions about spasmodic dysphonia, muscle tension dysphonia, and vocal tremor in October 2024. Responses were graded on a 5-point Likert scale on comprehensiveness, accuracy, appropriateness to patients, and patient safety. Readability was assessed with Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), Gunning Fog Index (GFI), and average word count. One-way ANOVA and Tukey HSD were used for statistical analysis; significance was set at p<.05.

Results: ChatGPT was significantly more comprehensive in its responses than both GAO (p < 0.01) and DI (p=.0357). Additionally, ChatGPT was significantly more accurate than both GAO (p <.01) and DI (p <.01), while GAO was more accurate than DI (p < 0.01). ChatGPT was found to be significantly more patient-friendly than both GAO (p <.01) and DI (p < 0.01). While ChatGPT had significantly safer recommendations than GAO (p <.01), its safety was similar to DI (p=.11). ChatGPT had a significantly higher FKGL than GAO (p <.01), and a lower FRES than both GAO (p <.01) and DI (p <.01), indicating a more difficult reading level. Nonetheless, all resources exceeded the recommended FKGL and GFI of less than 6 and FRES of 90-100. Interrater reliability was 0.8.

Conclusions: ChatGPT performed significantly better than GAO and DI in regards to comprehensiveness, accuracy, appropriateness, and was an overall safer AI-resource to use than GAO. All resources were above recommended reading grade level. AI models prove to be formidable alternative resources for patient centered clinical information.