(A944) Readability of Pediatric Otolaryngology Information: Comparing AI-Generated Content with Google Search Results

Introduction: This study aimed to compare the readability of pediatric otolaryngology information generated by GPT-4o with content retrieved from Google search results. By examining topics of varying complexity, we sought to assess the accessibility of medical content for pediatric patients' caregivers and provide insights into improving health literacy.

Methods: Ten pediatric otolaryngology topics were selected for analysis. Patient information articles were sourced from Google search results and generated by GPT-4o using standardized prompts. Readability was assessed using six metrics: Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), Gunning-Fog Index, Coleman-Liau Index, Automated Readability Index (ARI), and Simple Measure of Gobbledygook (SMOG). Texts were standardized to eliminate formatting biases before analysis. Statistical comparisons were performed to evaluate readability differences across topics.

Results: For simpler topics such as allergic rhinitis and otitis externa, GPT-4o content exhibited significantly higher FKGL scores and lower FRES scores than Google-derived content, indicating greater difficulty in readability. Differences diminished as topics became more complex, such as laryngomalacia and cleft lip and palate, with no statistically significant differences observed in readability metrics for these conditions. Across all topics, GPT-4o content consistently required a higher academic reading level for comprehension compared to Google content.

Conclusions: While GPT-4o content generally demonstrated greater difficulty in readability for less complex topics, its readability approached that of Google-derived content for more complex medical conditions. These findings highlight opportunities for optimizing AI-generated medical information to better meet the needs of diverse audiences, particularly caregivers seeking accessible health information.