Abstract
Introduction
The New Zealand Joint Registry (NZJR) monitors arthroplasty surgeon performance using revision rates presented on funnel plots. Historically this used all-time, unadjusted revision rate. Potential outliers are identified and required to undertake further audit of their results and present these in a peer-review setting. The method used by NZJR to assess surgical competence had not been validated. The aims of this thesis were to (i) validate the current method to audit surgical competence in hip and knee arthroplasty surgeons, (ii) identify potential improvements to this method, (iii) evaluate the feasibility of implementing these potential improvements using NZJR data and (iv) evaluate the impact of these potential improvements on which surgeons are identified as having outlier surgical performance.
Methods
Comparison with other organisations involved in evaluating surgical competence, and narrative review of the literature, were used to validate aspects of the NZJR approach to monitoring performance and identify potential improvements. A systematic review was undertaken to establish proof-of-concept that surgeon-level feedback can improve subsequent outcomes. A survey of New Zealand arthroplasty surgeons was completed to assess the face validity of monitoring surgical performance using revision rate as an indicator of competence and the identified potential improvements. NZJR data was then used to evaluate the feasibility of potential improvements, and the impact on which surgeons are identified as outliers.
Results
Comparison with other registries and literature review validated the use of revision rate as an indicator of surgical competence and funnel plots to identify outliers. Potential improvements included (i) reporting more recent timeframes which better reflect current practice, (ii) risk adjustment of revision rates to ensure fair comparisons between surgeons and (iii) considering a role for patient reported outcome measures (PROM) in monitoring performance to provide a more comprehensive understanding of surgeon outcomes. The systematic review established surgeon-level feedback can improve outcomes, in addition to providing quality assurance.
The survey received an excellent response. There was support for monitoring surgeon-level performance and using revision rate for this purpose. Surgeons supported the three identified potential improvements.
Revision at two years was identified as a feasible shorter reporting timeframe for both total hip and knee arthroplasty. This resulted in fewer outliers compared with the all-time timeframe with limited overlap in the identified outlier surgeons. Logistic regression models were developed using NZJR variables. These were poor fitting and additional covariates could not be obtained. Unadjusted revision rates continued to be used in this thesis. Funnel plots were developed for monitoring performance using six-month Oxford score, an arthroplasty specific PROM. There was minimal overlap in outliers identified using Oxford scores and revision rates.
Conclusions
This thesis validates many aspects of the NZJR approach to monitoring surgeon performance. Reporting a shorter timeframe, alongside the longer-term timeframe, and including six-month Oxford scores would be feasible and result in important differences in identified outliers. It is recommended NZJR considers implementing these improvements. Further variables are needed to develop adequate risk-adjustment models. It is recommended revision rates remain unadjusted, but NZJR should consider collecting a wider range of covariates to facilitate a risk-adjustment model in the future.