(A1008) Gender Bias in Artificial Intelligence-Written Letters of Reference for Otolaryngology Residency Candidates

Monday, October 13, 2025

12:00 PM - 1:00 PM EDT

Location: OTO EXPO, Posters

Has Audio

Presenting Author(s)

Grace Young, BS

Medical Student
Grace Young
Bloomington, Indiana, United States

Senior Author(s)

Janice L. Farlow, MD, PhD

Assistant Professor
Indiana University
Indianapolis, IN, United States

Co-Author(s)

Marianne Abouyared, MD

Associate Professor
University of California, Davis
Sacramento, California, United States
Alexandra E. Kejner, MD

Associate Professor
Medical University of South Carolina Rutledge Tower Annex
CHARLESTON, South Carolina, United States
LY

Linda X. Yin, MD

Assistant Professor
Mayo Clinic
Rochester, Minnesota, United States
RP

Rusha J. Patel, MD

Associate Professor
University of Oklahoma Health Sciences Center
Oklahoma City, Oklahoma, United States
Heather A. Edwards, MD

Director of Head and Neck Surgery, Associate Professor of Otolaryngology - Head and Neck Surgery
Boston Medical Center
Boston, MA, United States

Disclosure(s):

Grace Young, BS: No relevant relationships to disclose.

Janice L. Farlow, MD, PhD: No relevant relationships to disclose.

Marianne Abouyared, MD: No relevant relationships to disclose.

Alexandra E. Kejner, MD: No relevant relationships to disclose.

Linda X. Yin, MD: No relevant relationships to disclose.

Rusha J. Patel, MD: No relevant relationships to disclose.

Introduction: Written letters of reference (LORs) are an important component of the residency application process, and human-written LORs have been shown to contain gender-bias. Given that AI tools such as ChatGPT are increasingly utilized to draft LORs, it is important to understand how bias may be perpetuated in these tools. In a previous study, we identified gender bias in AI-written LORs when using prompts with randomly-generated resume variables. We sought to investigate whether this bias persisted using real applicant experiences, and how this compared to the LORs written by otolaryngology faculty.

Methods: We obtained 46 LORs for otolaryngology residency applicants written by faculty from 5 different institutions who regularly compose LORs. Prompts describing the candidate’s experiences using the exact phrasing as the letter writers were provided to ChatGPT4.0 in individual sessions. The writer-generated and AI-generated letters were compared using a gender-bias calculator which reports the ratio of male-associated ‘ability’ words to female-associated ‘grindstone’ words.

Results: On average, both faculty-written and AI-generated LORs showed male bias (18.7% and 37.2%, respectively). A paired t-test revealed significantly higher male bias in AI-generated LORs (p=0.0001). No significant difference in gender bias was observed between male and female applicants in either human- or AI-generated LORs. However, variability was significantly lower in AI-generated letters (p=0.0011), with all AI letters exhibiting male bias. Of the LORs, 54.3% were for male candidates.

Conclusions: While the use of AI for letter drafting resulted in overall male-bias, there was not a significant difference between letters using male versus female names, and the results did not vary as much as human-written letters. This suggests that AI-drafts could help reduce gender discrepancies. Further research is necessary to explore the broader implications of AI-assisted letter writing in residency selection, particularly in non-technical contexts.