Abstract
Background: Health information technologies (HIT), particularly computerised physician order entry (CPOE) systems with integrated clinical decision support (CDS), have enabled significant reductions in prescribing errors and improvements in prescribing quality. However, suboptimal and excessive implementation of CDS, especially interruptive alerts, has often limited effectiveness and introduced new issues such as alert fatigue. Evaluating the performance of implemented alerts is essential to ensure they are functioning as intended and to identify opportunities for optimisation. Routinely collected healthcare data generated by HIT offers a valuable resource to support these evaluation and optimisation efforts.
Aim: This thesis broadly aimed to develop data-driven strategies to evaluate and optimise CDS alerts for inpatient prescribing in MedChartTM, the electronic medication management (eMM) system used in the Waitaha Canterbury health district of New Zealand. Specifically, it sought to: (1) highlight the importance of evaluating and optimising prescribing-related CDS; (2) demonstrate how routinely collected healthcare data can be leveraged to evaluate and optimise interruptive CDS alerts; and (3) identify opportunities to improve the reporting of methodology in published quantitative evaluations of interruptive CDS alerts for inpatient prescribing. Through this work, the thesis aimed to generate practical insights to inform the governance of prescribing-related CDS alerts both locally and internationally.
Methods: In the thesis, a variety of methods were used, predominantly quantitative in nature. In Chapter 2, 16 published test scenarios were used to perform vulnerability testing of MedChartTM by simulating common, high-risk prescribing errors. The difficulty of completing the test scenarios was measured using a 5-point Likert scale and compared to the results from other hospital systems. Chapters 3, 4, and 5 were retrospective cohort and pre-post studies that used routinely collected healthcare data. Data came from the local data warehouse and MedChartTM data tables, and included patient demographics, inpatient prescriptions, discharge diagnoses, and alert data spanning from 2017 to 2024. By linking and analysing these datasets, the studies evaluated the impact of having alerts, not having alerts, and optimising alerts on alert burden, alert effectiveness, prescribing behaviour, and clinical outcomes. A scoping review was conducted in Chapter 6 to examine the reporting of alert metrics and electronic alert data extraction methods in published studies evaluating interruptive alerts for inpatient prescribing.
Results: In Chapter 2, vulnerability testing of Waitaha Canterbury’s MedChartTM revealed that: (1) it had strong protections against certain prescribing errors but remained vulnerable to other error types, (2) it shared many of the vulnerabilities observed in other hospital systems, and (3) the system’s ability to protect against certain errors varied by prescribing workflows and workarounds used by clinicians.
In Chapter 3, implementation of an interruptive CDS alert designed to promote laxative co-prescribing with clozapine was associated with increased co-prescribing of regularly scheduled non-bulking laxatives (OR, 1.3; 95% CI, 1.0–1.8; p = 0.035) and any laxatives (OR, 3.5; 95% CI, 2.1–5.8; p < 0.001). The alert was also associated with improved timeliness of laxative co-prescribing. However, the small number of coded constipation events precluded a rigorous assessment of associated changes in inpatient constipation rates.
In Chapter 4, the absence of penicillin-cephalosporin cross-reactivity alerts was associated with higher rates of cephalosporin prescribing in patients with penicillin adverse drug reaction labels compared to those without such labels (OR, 12.5; 95% CI, 10.6–14.9, p < 0.001). This was expected to be associated with reduced use of less safe and less effective non-penicillin, non-cephalosporin antibiotics.
In Chapter 5, the reconfiguration of four antithrombotic duplicate prescribing alerts to prevent triggering for stat dose prescriptions was associated with a 29.0% overall reduction in alert rate. The reconfiguration was also associated with a significant increase in the number of alerts associated with the cessation of an antithrombotic duplication from 32.8% to 44.5% (p < 0.001). Reconfiguration was associated with no significant changes in antithrombotic co-prescribing patterns.
In Chapter 6, only 40% of contemporary evaluations of interruptive alerts for inpatient prescribing included distal (post-alert) metrics of alert effectiveness. Furthermore, the metrics used to quantify alert burden and distal effectiveness were highly heterogeneous. For alert data extraction, only 21% of studies specified the data extraction tool used, and 42% reported the data variables being extracted.
Conclusion: This thesis demonstrates the importance of evaluating and optimising CDS alerts for inpatient prescribing and highlights the value of routinely collected healthcare data in supporting these processes. The thesis also highlights some current challenges in quantitative evaluations of alert performance that need to be addressed in future research. This work offers a key piece of a practical blueprint for data-driven governance and improvement of prescribing-related CDS alerts in hospitals.