(A245) The Landscape of ACGME Mandated Assessments of Surgical Skills: A Document Analysis

Introduction: Requirements for assessing resident’s surgical skills vary across surgical specialties and represent opportunities for cross-specialty adoption of best practices. However, the landscape of procedural skills assessments in graduate medical education is broad and complex. To map the current landscape of Accreditation Council for Graduate Medical Education (ACGME) requirements, we systematically compared the requirements in otolaryngology-head and neck surgery (OHNS) with those in other surgical fields.

Methods: We performed a qualitative document analysis to examine specialty-specific ACGME documents (common program requirements, case log requirements, milestones requirements). Findings were iteratively coded and themes were generated.

Results: Documents from nine ACGME surgical specialties were included. All specialties mandate case logs as a measure of surgical experience and six specialties, including OHNS, stipulate minimum case log numbers for select procedures or procedure categories. All specialties log resident roles (e.g., assistant surgeon) other than plastic surgery. Seven specialties mandate programs to provide access to surgical simulation lab, but only OHNS and ophthalmology specify the assessment of skills in these settings. OHNS requires annual evaluations of surgical competence but does not specify the format of assessments—a limitation shared by most other specialties. Within the milestones, OHNS assesses residents on nine procedural-based topic areas (three of which include specific examples with procedure type and performance level), compared to an average of 5.5 procedure-based topic areas across other specialties. No specialty mandates the use of specific standardized tools such as Objective Structured Assessments of Technical Skills (OSATS) or workplace-based assessments.

Conclusions: There is a lack of specificity in mandated skill assessments for OHNS and other surgical specialties, highlighting the potential need for validated, standardized methods to evaluate surgical competencies across specialties.