Geometric accuracy and time efficiency evaluation of a deep learning auto-segmentation system for head and neck radiotherapy: a single-center validation study

Background: Automated segmentation in radiotherapy is critical for workflow efficiency, yet commercial prototypes require rigorous validation before deployment. This study aims to validate the geometric accuracy and time efficiency of a prototype deep learning contouring (DLC) system for organs at risk (OARs) in a cohort of head and neck cancer (HNC) patients.

Methods: Fifty consecutive patients receiving intensity-modulated proton therapy (IMPT) for HNC between September 2019 and January 2021 at a single tertiary referral center in Taiwan were enrolled. Manual contours (MC) defined by one of three radiation oncologists served as the reference standard. We evaluated three strategies: unmodified DLC (DLC-u), DLC with manual modification (DLC-m), and MC. Performance was assessed using the Dice similarity coefficient (DSC), Hausdorff distance (HD), and a 5-point subjective clinical usability scale. Time efficiency was compared using paired t-tests.

Results: DLC-u demonstrated robust geometric concordance for large, high-contrast organs such as the eyes, brainstem, and parotid glands with mean DSC ≥0.80. In contrast, performance for small or low-contrast structures was suboptimal, with the optic chiasm and pharyngeal constrictor muscle (PCM) achieving mean DSCs of 0.44 and 0.42, respectively. The correlation between objective metrics and subjective scores was modest (|τ| ≤0.42). The human-in-the-loop workflow (DLC-m) significantly reduced total contouring time from 35.3 to 24.7 minutes (30.0% reduction across all OARs; P<0.001), corresponding to a 48.6% reduction (21.8 to 11.2 minutes; P<0.001) when restricted to the structures covered by the DLC model.

Conclusions: The evaluated DLC system significantly reduced the time required for OAR contouring. However, due to suboptimal performance in critical neural structures and the mandible, the system should be implemented as a supportive tool, necessitating careful expert review.