(A514) Can Artificial Intelligence Correctly Differentiate between Vagal Schwannomas and Sympathetic Chain Schwannomas?

Introduction: To determine the ability of artificial intelligence (AI) ChatGPT to accurately differentiate between vagal and carotid schwannomas from contrast-enhanced MRI images and CT images.

Methods: Study

Design: Comparative analysis.

Methods: A variety of peer reviewed studies from reputable journals in otolaryngology were accessed. Four MRI images, two of vagal schwannomas and two of sympathetic chain schwannomas, and four CT images, two of each diagnosis, were extracted. Each image was independently entered into ChatGPT 4o. The AI model was then prompted to provide the correct diagnosis. Accuracy of responses were graded based on the designations provided in the original sources. Univariate analyses were used to identify statistical associations.

Results: Eight images, four of each diagnosis, were given to ChatGPT. The AI model had an overall accuracy of 62.25%. Notably, GPT diagnosed vagal schwannoma at a greater rate than sympathetic chain schwannoma (75.00% of the time). When stratified by image modality type, GPT correctly diagnosed three of the MRI images (75.00%) and two (50.00%) of the CT images. On Fisher's exact test, there was no significant difference between overall accuracy of MRI images compared to CT images (p > 0.05).

Conclusions: While ChatGPT is able to recognize schwannomas in CT and MRI, the AI model does not have great accuracy in differentiating between vagal schwannomas and sympathetic chain schwannomas. Further refinement is necessary for reliable clinical application.