Skip to main content
AAO-HNSF 2025 Annual Meeting & OTO EXPO Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Scientific Oral Presentations

Rhinology/Allergy

Session: Scientific Oral Presentation 2 - Rhinology and Allergy

Development of an Operative Competency Assessment Tool for the Endoscopic Endonasal Transsellar Approach

Sunday, October 12, 2025
1:54 PM - 2:00 PM EDT
Location: 125
Disclosure(s):
Aviv Spillinger, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Garret W. Choby, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Blair M. Barton, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Mark B. Chaskes, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Ryan A. Rimmer, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Lauren M. North, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Victoria S. Lee, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Edward C. Kuan, MD, MBA:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Sandra Y. Lin, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Introduction: Surgical training has undergone a paradigm shift from traditional time-based models to competency-based medical education. This study aims to develop and validate a video-based instrument to assess the technical competence of trainees performing endoscopic endonasal transsellar approaches.

Methods: A prospective study design (March 2023 – January 2025) was employed. An Objective Structured Assessment of Technical Skills (OSATS)-based evaluation tool for the endoscopic endonasal transsellar approach was created, consisting of a Task-Specific Checklist (TSC) and a Global Rating Scale (GRS) rated on a five-point Likert scale. Endoscopic endonasal transsellar procedures performed by residents on live patients in the operating room were video recorded. Videos were then distributed electronically to attending-level skull base surgeons for rating using the OSATS tool. Evaluators were blinded to the residents’ identities and postgraduate year (PGY) levels. Construct validity was assessed by comparing mean TSC and GRS scores across PGY levels using analysis of variance (ANOVA). Internal consistency and interrater reliability were measured with Cronbach’s alpha and the intraclass correlation coefficient (ICC), respectively.

Results: A total of 89 assessments were completed by nine raters for seven residents (PGY levels 3–5). It took on average 13 minutes (range: 4–30 minutes) to review and evaluate each video. The instrument demonstrated excellent internal consistency (Cronbach's alpha = 0.96) but low interrater reliability (ICC = 0.3). Construct validity was confirmed, with higher PGY levels associated with improved performance: mean TSC scores were 3.3, 3.5, 3.8 (p = 0.07), and mean GRS scores were 2.8, 3.2, 3.8 (p = 0.03) for PGY 3, 4, and 5, respectively.

Conclusions: This video-based assessment tool for evaluating technical competence in the endoscopic endonasal transsellar approach demonstrates excellent internal consistency and construct validity, with higher PGY levels correlating with improved performance. Further refinements are needed to improve interrater reliability.

    Presenting Author(s)

  • Aviv Spillinger, MD

    Resident
    University of Wisconsin
    Madison, WI, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

    • Author(s)

  • QY

    Qiuyu Yang, MS

    University of Wisconsin, United States

    Disclosure information not submitted.

  • ER

    Erin K. Reilly, MD

    Erin K. Reilly
    Thomas Jefferson University Hospital
    Philadelphia, Pennsylvania, United States

    Disclosure information not submitted.

  • GC

    Garret W. Choby, MD

    Associate Professor of Otolaryngology and Neurological Surgery
    University of Pittsburgh School of Medicine
    Pittsburgh, PA, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • Blair M. Barton, MD

    Assistant Professor
    Ochsner Medical Center
    New Orleans, LA, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • MC

    Mark B. Chaskes, MD

    Assistant Professor
    Northwell Health/Zucker School of Medicine
    Westbury, NY, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • Ryan A. Rimmer, MD

    Assistant Professor of Surgery
    Division of Otolaryngology, Department of Surgery, Yale School of Medicine
    Cheshire, CT, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • LN

    Lauren M. North, MD

    Assistant Professor
    Medical College of Wisconsin
    Milwaukee, WI, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • CR

    Christopher R. Roxbury, MD

    Associate Professor
    University of Chicago Medicine
    Cleveland, OH, United States

    Disclosure information not submitted.

  • Victoria S. Lee, MD

    Associate Professor of Otolaryngology
    University of Illinois Chicago
    Chicago, IL, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • Edward C. Kuan, MD, MBA

    Professor
    University of California, Irvine
    Orange, CA, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

  • SL

    Sandra Y. Lin, MD

    Professor and Chair of Otolaryngology - Head & Neck Surgery
    University of Wisconsin School of Medicine and Public Health
    Madison, WI, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

    • Senior Author(s)

  • IK

    Ian Koszewski, MD

    University of Wisconsin, United States

    Disclosure information not submitted.