Objective Structured Assessment of Technical Skills in Mastoidectomy Using Three-Dimensional Printed Temporal Bone Models

Introduction: Mastoidectomy is a core otolaryngology procedure requiring technical precision. Traditional skill assessment tools utilize cadaveric temporal bones, which have limitations related to anatomical variability and accessibility. Three-dimensional (3D) printed temporal bone models offer standardized anatomy and allow consistent and anonymous assessment. While used for training, their role in evaluating skills remains underexplored. This study integrates 3D-printed temporal bone models into an objective structured assessment of technical skills tool (OSAT) to evaluate mastoidectomy performance.

Methods: Over a three-year period, otolaryngology residents performed mastoidectomies on 3D-printed temporal bones. Three expert faculty members anonymously evaluated each performance using a validated 14-item Likert scale for final product analysis. Statistical analyses included t-tests, Mann-Whitney U, Wilcoxon signed-rank tests, and Intraclass Correlation Coefficients (ICC) calculations.

Results: Thirty-two residents participated, with a total of 64 3D-printed temporal bones assessed. Median scores improved steadily across training levels, from 35.0 in postgraduate year (PGY) 1 to 53.7 in PGY 5 out of 70. Linear regression showed a positive relationship between training duration and performance. Senior residents (PGY 4-5, median score 54.0) significantly outperformed junior residents (PGY 1-3, median score 42.7; P = .0035). Across all training levels, median scores improved from 43.2 on the first attempt to 52.5 on the third, though not statistically significant. The assessment tool demonstrated excellent inter-rater reliability by the three expert evaluators (ICC = 0.92, P < .0001).

Conclusions: The consistency of 3D-printed temporal bones, judged by final product analysis, facilitates fair, objective, and reliable assessment for formative and summative evaluations. By longitudinally tracking skill improvement and differentiating performance across training levels, this OSAT supports the transition to competency-based surgical education.