Patients Use of Large Language Models in Otolaryngology: Safety Concerns and Ethical Responsibilities for Physicians?

Introduction: Large language models (LLMs) are increasingly used by patients to access health-related information. While these tools offer accessibility and scalability, their accuracy and safety remain under scrutiny. This study aims to evaluate the reliability of LLMs from the patient’s perspective, to facilitate the responsibility of physicians to guide patients in their use.

Methods: A systematic review of the literature was conducted using PRISMA guidelines. Databases including PubMed, Scopus, and Embase were searched for studies published between 2023 and 2024. Articles addressing LLM use for health information were screened, and data were extracted regarding the accuracy of LLM-generated advice, associated risks, patient perspectives, and ethical considerations.

Results: Of 426 articles, 71 studies were included. LLMs demonstrated 86.4% accuracy in preoperative education (n=14) for common procedures like otosclerosis surgery but exhibited misinformation in addressing complex risks, such as following Head & Neck surgeries. Postoperative guidance (n=17) showed 81.7% accuracy, improving patient comprehension and emotional well-being post-surgery. For otolaryngology general questions (n=40), such as Ménière’s disease, LLMs achieved 69.54% accuracy, with the best results (accuracy of 82.4%) in general otolaryngology (such as tonsillectomy) and worse in Head & Neck cancer (57.86%).

Conclusions: LLMs hold promise for easy and accessible patient education, particularly for straightforward otolaryngology conditions and procedures, yet their limitations in addressing complex medical scenarios necessitate caution. Physicians should proactively guide patients on the limits of LLM use, advocating for their responsible integration into healthcare. This highlights the need for further refinement of LLMs to improve their reliability and ensure safe, ethical applications in otolaryngology.