Abstract
The translation of AI-generated brain metastases (BM) segmentation into
clinical practice relies heavily on diverse, high-quality annotated medical
imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing
and benchmarking algorithms using rigorously annotated internationally compiled
real-world datasets. This study presents the results of the segmentation
challenge and characterizes the challenging cases that impacted the performance
of the winning algorithms. Untreated brain metastases on standard anatomic MRI
sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets
were annotated in stepwise method: published UNET algorithms, student,
neuroradiologist, final approver neuroradiologist. Segmentations were ranked
based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives
(FP) and false negatives (FN) were rigorously penalized, receiving a score of 0
for Dice and a fixed penalty of 374 for HD95. Eight datasets comprising 1303
studies were annotated, with 402 studies (3076 lesions) released on Synapse as
publicly available datasets to challenge competitors. Additionally, 31 studies
(139 lesions) were held out for validation, and 59 studies (218 lesions) were
used for testing. Segmentation accuracy was measured as rank across subjects,
with the winning team achieving a LesionWise mean score of 7.9. Common errors
among the leading teams included false negatives for small lesions and
misregistration of masks in space.The BraTS-METS 2023 challenge successfully
curated well-annotated, diverse datasets and identified common errors,
facilitating the translation of BM segmentation across varied clinical
environments and providing personalized volumetric reports to patients
undergoing BM treatment.