Abstract
Objectives: To develop and validate a framework combining deep learning-based detection with geometric severity assessment of open gingival embrasures (OGEs) from intraoral photographs.
Methods: A total of 3995 OGEs from 653 intraoral photographs collected across three orthodontic centres were annotated at the pixel level for maxillary tooth crowns and anterior OGEs. Model development used five-fold cross-validation on images from Centres 1 and 2, with patient-level separation to prevent data leakage. Five cross-validation models per architecture were evaluated on an independent test set from Centre 3. Performance was evaluated at the pixel (DSC, IoU), instance (precision, recall, F1-score), and grading level (accuracy, AUC).
Results: Mask R-CNN achieved superior OGE segmentation performance (DSC 0.812 ± 0.111 vs. 0.597 ± 0.135 for YOLOv8), with improved boundary accuracy, while both models demonstrated high tooth crown segmentation accuracy (DSC > 0.95). Among successfully detected and matched OGEs, the grading framework demonstrated strong discriminative performance, with area under the curve values of 0.939 ± 0.009 for YOLOv8 and 0.933 ± 0.008 for Mask R-CNN, and comparable classification accuracy between models (0.86 ± 0.01 for YOLOv8 and 0.86 ± 0.03 for Mask R-CNN). As no Grade III OGEs were present in the external test set, grading performance was evaluated only between Grades I and II.
Conclusions: This study presents a framework combining deep learning-based OGE detection with interpretable geometric severity grading. Grading performance was evaluated only between Grades I and II in the external test set; Grade III requires further validation.
Clinical significance: The proposed approach provides objective OGE assessment from clinical intraoral photographs and may support future remote orthodontic monitoring after further validation with patient-acquired images.