Abstract
Introduction
Effective planning and organising of the health workforce are important for optimal patient outcomes, a healthy work environment for staff and improving service efficiency. The allied health workforce (social workers, speech and language therapists, physiotherapists, dietitians, occupational therapists, and many other professions) in Aotearoa New Zealand (NZ) has not previously had a specific way of being planned and organised. Health New Zealand (Health NZ) have developed a set of method and processes to do this – the 'Health NZ allied health staffing methodology'. The methodology has been piloted for a group of allied health professions but needed testing before rolling out more widely.
Therefore, this evaluation project aimed to:
1. Understand if the methodology resulted in the right number and type of staff being in the right place. To do this, the project undertook to:
a. Assess how well suited the methodology is to the workplaces and professions in which it is being used.
b. Assess how well the methodology works in practice to achieve the best patient outcomes, work environment and service efficiency, including to advance equity.
2. Identify if and how the methodology should be modified before its wider roll out.
Current Health NZ allied health staffing methodology
The Health NZ allied health staffing methodology involves allied health clinicians entering data on their workloads during their work (activity data). These are used in a calculation formula to work out the required full-time equivalent (FTE) allied health staff. The output is weighed up against other information including trends in the activity data and outcomes collected for patients, staff and services. These findings are brought together to assess two key questions: Do we have enough staff? And do we have the right skill and staff mix?
Evaluation Approach
Evaluation methodology
This evaluation used qualitative and quantitative research methods to test a theoretical model of the methodology for how it worked and in which contexts - a "realist" evaluation.
Phase one involved interviews with 18 organisational stakeholders and a scoping review of the literature to develop the theoretical model of the staffing methodology.
Phase two was a qualitative study in which we interviewed 56 allied health staff, leaders and organisational experts, about their perspectives on the methodology.
Phase three looked at the data collected by districts. We assessed how consistent and complete this data is. Where possible, we examined if the staffing methodology had resulted in any changes for patient, staff and service outcomes and equity.
In the fourth phase, we brought together the qualitative and quantitative findings to see where the methodology appeared to have benefit, and why. This was reviewed by 23 expert panellists who participated in either a Māori panel. Pacific panel, or general panel. Through discussion and agreement with the panellists, recommendations were made for how the staffing methodology should be modified.
Conclusions
The Health NZ allied health methodology involves a large amount of data, all of which needs to be consistent, complete, and reliable, i.e. of high quality.
The research found that the quality of the data and the methodology processes were not what they needed to be. There were several reasons for this. A poor match between the methodology processes and how allied health work was one reason. Differences in how data was collected and interpretations of what variables meant was another. Inadequate data and digital infrastructure and support was a third. These issues meant that staff found data entry to be a lot of work, and there was a limited ability to extract, check, or use data.
Regarding the FTE calculation, we found that the calculation's use of 1-FTE contracted hours rather than 1-FTE clinical hours in the final transformation step did not accurately estimate the allied health staff required.
An additional observation was that the inability of staff and services to use their data was adding to staff burnout.
Recommendations for future rollout of the methodology are:
• Replace the routine manually collected activity data with targeted, periodic well defined work sampling using nationally standardised tools and processes.
• Develop a regional or (ideally) national electronic health record that can automate allied health clinical activity data collection and extraction, with careful attention on data quality and monitoring.
• Refocus on payroll and referral data as the most clear, well-understood variables.
• Adapt decision-making to more formally include local knowledge and professional judgement, by combining triangulation with a deliberative committee approach.
• Implement widespread, routine collecting of patient outcomes.
• Amend the FTE calculation to use 1-FTE clinical hours in the final transformation.
• Place equity at the centre of the methodology and operationalise this by routinely separating out outcomes for priority groups and recognising cultural work.
• Invest in robust data and digital systems. Educate and support staff and services to extract, collate, analyse and use the data.
Taking these steps would establish a reliable platform for the allied health staffing methodology to evolve.