Abstract
Infertility affects between 8% and 15% of couples of reproductive age worldwide, with rates in New Zealand estimated at 12.5%. Of those affected, 68% seek medical help, often through assisted reproductive technologies (ART). For many individuals, this process presents substantial emotional and financial burdens. Development of reliable predictive modelling of ART success could help clinicians provide informed, personalised treatment support for patients, aiding and easing these burdens. Modern statistical and machine learning approaches were combined to develop a pipeline for estimating ART outcomes. The pipeline was developed within the R statistical package using entirely original code and built from an independent, fabricated dataset that modelled outcomes based on fertility information in the literature.
At the first stage of the pipeline, data were numerically encoded using one-hot encoding before splitting the dataset into training (70%) and testing (30%) sets. To handle class imbalance, Synthetic Minority Oversampling Technique (SMOTE) was applied to the training set. Least Absolute Shrinkage and Selection Operator (LASSO) regression was used to identify important linear variables, while Generalised Additive Models (GAM) were used to identify significant non-linear predictor variables. Six machine learning algorithms were selected and incorporated into the pipeline. These were: Neural Networks (NN), Random Forests (RF), Decision Trees (DT), Gradient Boosting Machines (GBM), Extreme Gradient Boosting (XGBoost), and Support Vector Machines (SVM). Each algorithm was trained with the retained variables, and the performance was evaluated on the test set using AUROC, calibration, Brier score, and confusion matrices. The best performing machine learning algorithm was interpreted using Shapley Additive Explanations (SHAP), and the model was incorporated into a user interface using ShinyR. As an example of how the model could be clinically applied, the ShinyR interface includes the ability to enter unique values into the pipeline variables to investigate how these influence the estimation of ART outcomes.
This pipeline was applied to Fertility Associates data to develop and validate a predictive model estimating the probability of ART success prior to embryo transfer. The North Island cohort was used as the test dataset, and the South Island cohort was reserved for external validation. The XGBoost model outperformed other models, achieving a high AUROC on both the North Island test set (0.70) and on the South Island validation set (0.69). The high AUROC value observed during validation confirms its generalisability to new datasets. Imputation analyses showed that retaining more patients through imputation did not substantially reduce performance, with XGBoost still achieving a strong AUROC (0.68). SHAP was used to interpret the top predictors contributing to XGBoost’s predictions and probability estimates of treatment success, spanning both pre-cycle and in-cycle variables. Five patients were randomly removed from the dataset prior to data analysis. The model’s ability to predict the outcome for these patients was compared against the current clinical approach, which relies on expert clinical inferences. The model correctly predicted all five outcomes, compared against one correct clinical prediction.
This study represents the first New Zealand application of machine learning in the development of a predictive model for ART outcomes. It demonstrated that machine learning can generate accurate, interpretable models which can be investigated for clinical use. This predictive model represents a substantial advance in the current approach to helping guide treatment, reduce uncertainty, and advance personalised reproductive medicine in New Zealand.