The chapter discusses the importance and challenges of comparing health system performance across countries. While international comparisons are valuable for identifying strengths and weaknesses in healthcare systems, making improvements based on these comparisons is complex due to variations in governance, funding, resources, and service delivery across different countries. The chapter focuses on cluster analysis as a methodological approach to grouping and comparing health systems and discusses two main approaches to clustering: grouping health systems based on their overall characteristics, such as financing methods, or type of coverage; targeted policy-based clustering, with a focus on specific policy questions to help make actionable policy improvements.
How Do Health System Features Influence Health System Performance?
1. Opportunities and challenges for the comparison of health system performance based on their institutional characteristics
Copy link to 1. Opportunities and challenges for the comparison of health system performance based on their institutional characteristicsAbstract
There is a long history of comparing health system performance. International comparisons are recognised as being an important tool for assessing performance and prompting improvement (Papanicolas and Smith, 2013[1]; OECD, 2024[2]) as they can raise awareness of health systems’ relative strengths and shortcomings, facilitating international learning and stimulating needed policy debates.
Yet determining how to improve health system performance based on international comparisons is fraught with challenges (Bowden, Figueroa and Papanicolas, 2024[3]). Health systems can differ in many ways, for example in how they are governed, how they are funded, how they generate and deploy resources, and how they deliver services. While there is widespread agreement that these features influence health system performance, it is more difficult to assess how much they matter, which ones matter most, how they individually affect different dimensions of performance, and how they are affected by the wider context in which they operate – be that the rest of the health system, other country-specific factors or the social determinants of health.
Data on factors that affect performance – such as health system features, the demographic characteristics of the populations being compared and the broader political, socio‑economic and cultural context within countries – can be used to make more meaningful empirical comparisons through using them to identify comparators (groups or clusters of health systems), to adjust for exogenous variation or to help interpret results (Jacobs, Smith and Street, 2006[4]; Papanicolas and Marino, 2024[5]).
Cluster analysis (James et al., 2021[6]; Hastie, Tibshirani and Friedman, 2009[7]) is a useful descriptive tool that can be used to group health systems as it provides valuable insights into a dataset where there are distinct populations, which might not be seen by simply exploring distributions and comparing data parameters. For example, it has been recently used by OECD to assess the transferability of public health interventions (Wiper et al., 2022[8]).
Various approaches have been developed to cluster health systems in the past (Ferreira et al., 2018[9]). These generally use range of data on the characteristics of health systems (and sometimes their capacity, performance, and other factors) to identify typologies (clusters) based on the overall design or key features of health systems. This can include data on more or less modifiable characteristics – e.g. revenue sources versus payment systems – depending on the approach and aims of the researchers. As an example, Reibling et al. (2019[10]), with the objective of identifying groups or clusters of peers, developed a typology of five healthcare systems based on supply, public/private mix, access regulation, primary care orientation and performance. Ferreira et al. (2018[9]) used three factors – an aggregate of health systems financing, medical doctors per 100 000 population and hospital discharges due to diabetes, hypertension or asthma per 100 000 population – to identify five clusters of countries. Gabani et al. (2023[11]) used health expenditures by financing scheme as a share of total health expenditure to identify three clusters of countries – those where health expenditure is channelled predominantly via a government-funded arrangement, a contributory social health insurance arrangement or an out-of-pocket arrangement. Paris et al. (2016[12]) use the type of primary coverage to identify four clusters of health systems: residence‑based; contributory, single payer; contributory, multiple insurers with automatic affiliation; contributory, multiple insurers with choice of insurer.
Another approach is to cluster health systems based on more targeted policy questions (Papanicolas et al., 2024[13]). This could include how countries use a particular policy lever (e.g. payment systems) or how they approach a particular policy problem (e.g. regulating new technology). This approach has the potential to offer more actionable insights for policy improvement.
Both of these approaches to clustering are valid depending on the question being asked and the intended use of the comparison. But more targeted approaches are underused and could be developed using new, richer, data such as the OECD’s Health System Characteristics (HSC) survey (see Annex A), the European Observatory’s Health System in Transition profiles and the WHO Health Financing Progress Matrix.
The aim of this work is to understand the links between health systems overall design and performance of OECD countries on key health system indicators. New OECD data on health system characteristics were used to develop updated clusters of health systems based on their overall design and policy approach. Health system performance was compared between and within these clusters to understand potential links between overall health system policies and efficiency. A more targeted approach was then taken to clustering and performance comparison based on selected policy questions – including how payment systems might shape performance, and differences in performance between more or less primary care oriented systems – to help identify actionable policy levers that – independently of the overall design of a health system – could improve performance.
References
[3] Bowden, N., J. Figueroa and I. Papanicolas (2024), “Bridging borders: current trends and future directions in comparative health systems research”, Health Services Research, https://doi.org/10.1111/1475-6773.14385.
[9] Ferreira, P. et al. (2018), “EU health systems classification: a new proposal from EURO-HEALTHY”, BMC Health Services Research, Vol. 18/1, https://doi.org/10.1186/s12913-018-3323-3.
[11] Gabani, J., S. Mazumdar and M. Suhrcke (2023), “The effect of health financing systems on health system outcomes: A cross-country panel analysis”, Health Economics, Vol. 32/3, pp. 574-619, https://doi.org/10.1002/hec.4635.
[7] Hastie, T., R. Tibshirani and J. Friedman (2009), The Elements of Statistical Learning, Springer New York, New York, NY, https://doi.org/10.1007/978-0-387-84858-7.
[4] Jacobs, R., P. Smith and A. Street (2006), Measuring efficiency in health care, Cambridge University Press.
[6] James, G. et al. (2021), An Introduction to Statistical Learning, Springer US, New York, NY, https://doi.org/10.1007/978-1-0716-1418-1.
[2] OECD (2024), Rethinking Health System Performance Assessment: A Renewed Framework, OECD Health Policy Studies, OECD Publishing, Paris, https://doi.org/10.1787/107182c8-en.
[13] Papanicolas, I. et al. (2024), “Policy questions as a guide for health systems’ performance comparisons”, Bull World Health Organ, pp. 550-552, https://doi.org/10.2471/BLT.24.291635.
[5] Papanicolas, I. and A. Marino (2024), International comparisons: who has the best health system in the world?, Edward Elgar Publishing.
[1] Papanicolas, I. and P. Smith (2013), Health system performance comparison: an agenda for policy, information and research, Open University Press.
[12] Paris, V. et al. (2016), “Health care coverage in OECD countries in 2012”, OECD Health Working Papers, No. 88, OECD Publishing, Paris, https://doi.org/10.1787/5jlz3kbf7pzv-en.
[10] Reibling, N., M. Ariaans and C. Wendt (2019), “Worlds of Healthcare: A Healthcare System Typology of OECD Countries”, Health Policy, Vol. 123/7, pp. 611-620, https://doi.org/10.1016/j.healthpol.2019.05.001.
[8] Wiper, O. et al. (2022), “Cluster analysis to assess the transferability of public health interventions”, OECD Health Working Papers, No. 133, OECD Publishing, Paris, https://doi.org/10.1787/a5b1dcc1-en.