Abstract
There is a growing body of literature using multi-criteria decision analysis (MCDA) methods for prioritising health interventions. However, there has been very little application of MCDA to prioritise funding for research across health conditions, including non-communicable diseases (NCDs). NCDs are non-transmissible diseases that are not spread from person to person – e.g. diabetes mellitus, coronary heart disease, back pain, dementia and depression. Given limited resources, funding needs to be systematically allocated for research into the most pressing NCDs by explicitly identifying priorities for health research. Methods based on MCDA have attracted increasing attention by policy-makers and researchers by systematically forming and solving the multi-dimensional aspects of the decision problems, particularly in the health system. This thesis aims to investigate the use of MCDA to support health research funding across NCDs.
Following chapters of introduction and a review of commonly used methods, the thesis includes three main chapters (i.e. Chapters 3, 4 and 5), as well as the concluding chapter that provides policy implications and research contributions of the thesis. In Chapter 3, a widely-used MCDA method – i.e. the Potentially All Pairwise RanKings of all possible Alternatives (PAPRIKA) method administered through 1000minds software – is applied to create a priority list of NCDs to support health research funding. Informed by the literature, a set of prioritisation criteria – e.g. deaths, loss of quality-of-life and cost of the disease – is specified to evaluate NCDs in terms of their priority for health research funding decision-making. Their weights, representing their relative importance, are calculated based on a survey of stakeholders from various sectors of the New Zealand (NZ) health system.
The most important criterion for prioritising NCDs in terms of their overall burden to society (and hence their importance for health research funding) is ‘deaths across the population’ (mean weight = 27.7%), followed by ‘loss of quality-of-life across the population’ (23.0%), then ‘cost of the disease to patients and families’ (18.6%), ‘cost of the disease to the health system’ (17.2%) and the least-important criterion, ‘disproportionately affects vulnerable groups’ (13.4%). The criteria are used to rate NCDs based on evidence concerning their performance on the criteria. The rated NCDs are then ranked using the criteria mean weights from the survey. Each NCD’s total score is presented based on a 0-100% scale, where 100% indicated an NCD with the highest levels on all criteria, and 0%, an NCD with the lowest levels on all criteria. The NCDs ranking is categorised into four tiers: Priority 1 (very critical): coronary heart disease, back and neck pain, diabetes mellitus; Priority 2 (critical): dementia and Alzheimer’s disease, stroke; Priority 3 (high): colon and rectum cancer, depressive disorders, chronic obstructive pulmonary disease, chronic kidney disease, breast cancer, prostate cancer, arthritis, lung cancer; Priority 4 (medium): asthma, hearing loss, melanoma skin cancer, addictive disorders, non-melanoma skin cancer, headaches. The MCDA-based framework developed in Chapter 3 enables incorporating multiple criteria for evaluating a range of NCDs in terms of their priority for research funding and the involvement of a diverse range of stakeholders. This framework is hence more likely to generate a priority list that is more acceptable to the key stakeholders. This priority list shows that it is essential to incorporate the multi-dimensional nature – e.g. mortality, morbidity and health care costs – of NCDs when evaluating their priority and eligibility for health research funding.
In Chapter 4, PAPRIKA is compared with the most well known and thus, the most widely-used MCDA method – i.e. the Analytic Hierarchy Process (AHP) administered using Expert Choice software. AHP is considered as a benchmark among the MCDA methods by many MCDA practitioners. Both AHP and PAPRIKA are two prominent MCDA methods that have been used in many different fields and appeared in many publications. It is worthwhile to compare the two methods based on their theoretical foundations. Along with AHP and PAPRIKA, their associated decision-making software – i.e. Expert Choice and 1000minds – are considered in the evaluation framework. The findings indicate that AHP (and Expert Choice) and PAPRIKA (and 1000minds) use different theoretical foundations at different stages of the decision-making process from eliciting participants’ preferences to calculating the criteria weights and alternatives scores. As such, PAPRIKA uses choice-based pairwise comparison questions, whereas AHP uses ratio scale-based paired comparison questions to elicit participants’ preferences. AHP, unlike PAPRIKA, does not enforce the transitivity property.
In Chapter 5, an empirical framework is established to evaluate the performance of AHP (and Expert Choice) and PAPRIKA (and 1000minds) based on the NCD survey (the main subject of this thesis), as well as a second survey about smartphones. In the framework, a holdout choice task is employed for investigating the performance of AHP (and Expert Choice) and PAPRIKA (and 1000minds) to predict participants’ actual choices. The findings reveal that PAPRIKA (and 1000minds) is more likely to outperform AHP (and Expert Choice) based on both decision case studies about NCDs and smartphones. PAPRIKA could produce repeatable results over time and show higher validity to predict participants’ actual choices.
Given the diversity of MCDA methods and in the absence of a gold standard, evaluating the chosen MCDA method(s) based on their theoretical foundations may not be sufficient. An appropriate MCDA method is also required to produce robust results. Ideally, the theoretical and empirical frameworks developed in Chapters 4 and 5 could help MCDA practitioners consider a holistic approach to justify the choice of MCDA – which is the primary purpose of ‘step 7’ in the MCDA process – and choose a method that is developed on sound theoretical foundations and that generates robust results.