OECD assessed several case study interventions targeting mental health promotion and/or prevention (Table A A.1). Together, the case studies cover several OECD and non-OECD European countries.
Selected case studies represent strategic, high-priority interventions among policymakers in the OECD and EU27. A full description of the selection process is in Box A A.1.
Case studies were selected using the following hierarchical process:
Case studies were submitted by delegates to OECD’s Expert Group on the Economics of Public Health, which includes representatives from all 38 member countries
Case studies involved in European Joint Actions, in particular the Joint Action ImpleMENTAL that has implemented mental health interventions across Member States
Case studies previously defined as “Best Practice” by member countries, such as those listed on the EU Best Practice Portal (European Commission, 2021[1])
This section outlines two complementary frameworks used to assess case studies, both of which were developed by the OECD – the Best Practice Framework and the Transferability Framework. Limitations associated with the analysis are also discussed.
The Best Practice Framework outlines five criteria to assess whether an intervention is “best practice” – namely Effectiveness, Efficiency, Equity, Evidence‑base, and Extent of coverage (Table A A.2). A review of the academic and grey literature, existing best practice frameworks and feedback from delegates to OECD’s expert Group on the Economics of Public Health informed the selection of criteria.
An intervention can be awarded a “stamp of approval” against one or multiple criteria if it performs particularly well relative to similar interventions.
Up and coming interventions (i.e. those that show promise but have not yet collected any of their own data) can be awarded a “promising best practice” stamp of approval for relevant criteria. 
For a selection of case studies, effectiveness and efficiency were measured using OECD’s Strategic Public Planning for NCDs (SPHeP-NCD) microsimulation model. To analyse the health and economic impact of mental ill-health, the OECD has adapted the SPHeP-NCD microsimulation model to allow for the simulation of mental ill-health trends across OECD and EU27 countries from 2025 to 2050. An overview of the model is provided in Box A A.2.
The OECD SPHeP-NCDs model is an advanced systems modelling tool for public health policy and strategic planning. The model is used to predict the health and economic outcomes of the population of a country or a region up to 2050. The model reproduces a comprehensive set of key behavioural and physiological risk factors and their associated non-communicable diseases, including mental ill-health. For these analyses, the model covers 30 EU/EFTA countries, including all the EU27 Member States as well as Iceland, Norway and Switzerland.
For each modeled country, the model uses demographic and risk factor characteristics by age and gender-specific population groups from international databases. These inputs are used to generate synthetic populations, in which each individual is assigned a profile with a certain risk of developing a disease each year. Incidence and prevalence of diseases in a specific country’s population are calibrated to match estimates from international datasets.
The model produces yearly cross-sectional representations of the population that can be used to calculate health status indicators such as life expectancy, premature deaths (including as a result of suicide), disease prevalence and disability-adjusted life years using disability weights. Healthcare costs of disease treatment are estimated based on a per-case annual cost, which is extrapolated from national health-related expenditure data. Treatment cost for anxiety and depressive disorders vary by severity level. For instance, cost of depressive disorders of moderate severity are 5 times greater than that of mild severity, while cost of severe severity are 20 times higher than that of mild severity, based on König et al. (2023[2]).The additional cost of multi-morbidity, which is an important factor in the case of mental disorders, and the extra cost of end-of-life care are also considered. The labour market module uses relative risks relating disease status with absenteeism, presenteeism (where sick individuals, even if physically present at work, are not fully productive), early retirement and employment. These changes in employment and productivity are estimated in number of full-time equivalent workers and with other parameters contribute to calculate the impact on gross domestic product, by applying a Cobb-Douglas production function.
The model includes three leading mental health diseases: major depressive disorders (MDD) (including three different levels of severity: mild, moderate and severe); generalised anxiety disorders (GAD) and alcohol use disorders. Although these mental disorders represent over 72% of the total prevalence of mental health conditions across the EU27 and EEA countries, the model’s results should be viewed as conservative since 28% of conditions remain excluded. In addition, the model does not capture the burden suffered by the persons exposed to individuals with mental disorders, such as family and friends.
Mental health disorders are modelled via specific modules created for each disease. For Major Depressive Disorder (MDD) and Generalised Anxiety Disorder (GAD), each individual in the model is assigned a score on two international scales used for diagnosing the disease (i.e. PHQ‑8 and GAD‑7 scales, respectively). Distribution of both scores are modelled using micro-level data with a zero‑inflated beta regression using parameters age, sex, and country to model PHQ, and age, sex and PHQ to model GAD. The regression is performed in two steps where a logistic regression is used to predict the probability of zero and a beta regression is used to model the final score, taking into account the zero probability and scaling the score to between 0 and 1 to fit the distribution. Various models were tested to reproduce as closely as possible the observed PHQ (patient health questionnaire) and GAD score distributions, with the zero‑inflated beta producing the best results. To compute individual scores distributions are inverted using the partially fixed quantile approach (one part is fixed throughout the individual’s life while the remaining part is recalculated every two years). The approach has the dual advantage of maintaining continuity in trajectory over time and creating individuals more likely to experience recurrent relapses throughout their lifetime. Based on the score, each individual has a probability of developing MDD and GAD, with higher scores indicating a greater risk of mental disorder. The nineth question of the PHQ‑9 score is modelled separately. The probability of having suicidal thoughts – any yes response to question nine on suicidal thoughts (from occasionally to almost every day) – was estimated based on microlevel data and modelled as a function of PHQ‑8 score. When question 9 is present, individuals are at higher risk of self-harm. Alcohol use disorders, on the other hand, depend on the pattern and volume of alcohol consumption, with higher consumption corresponding to a higher risk of developing the condition. All modules are calibrated to match prevalence data for the simulated diseases.
For more information on the OECD SPHeP-NCDs model, see the SPHeP-NCDs Technical Documentation, available at: http://oecdpublichealthexplorer.org/ncd-doc.
Public health interventions are complex given they involve multiple stakeholders, often target heterogeneous groups, and have outcomes affected by various direct and indirect factors. Therefore, positive outcomes achieved in one setting aren’t necessarily transferable to a different setting.
OECD has developed a Transferability Framework to assist policymakers assess whether a best practice intervention can be transferred from where it has been implemented (i.e. best practice “owner setting”) to a different country/region (i.e. the “target setting”). Specifically, whether the desired outcomes achieved in the owner setting are achievable in the target setting (Trompette et al., 2014[3]; Burchett, Umoquit and Dobrow, 2011[4]).
The Transferability Framework includes four contextual factors that affect transferability:
Population context: covers population characteristics such as sociodemographic factors as well as broader cultural considerations
Sector-specific context: covers governance/regulation, financing, workforce, capital and access arrangements in the sector the intervention operates
Political context: political will from key decision makers to implement the intervention
Economic context: the affordability of the intervention in the target setting.
In each case study, indicators to assess transferability are grouped under one of these four contextual factors. For the case studies presented in this document, countries are allocated into a group based on how far the indicator’s value is from the best practice owner setting. This method is referred to as the “distance from reference country” and is explained in Box A A.3. In addition, OECD developed a clustering methodology to group countries according to their potential to transfer a best practice intervention (Box A.A.4).
Indicators were sourced from international databases to maximise coverage across OECD and non-OECD European countries (e.g. OECD Health Statistics, Eurostat, World Bank Indicators, and the WHO). Policy indicators were largely retrieved from the OECD/WHO Regional Office for Europe (2023), Mental Health Systems Capacity Questionnaire 2023 (OECD/WHO Regional Office for Europe, 2023[5]). Relevant indicators were excluded if data was missing for the best practice owner setting and could not be identified through desktop research, or, if more than 50% of data was missing across countries.
By using international data, the scope of the analysis was inevitably limited – i.e. indicators from international sources are high-level and don’t cover all relevant information for assessing transferability. Therefore, each case study also includes a set of “new indicators” (i.e. those with no publicly available information) policymakers should consider before transferring the intervention.
Finally, indicators to measure the risk factor level in each country (e.g. obesity rates) were not included given it is presumed all OECD and non-OECD European countries face challenges caused by growing rates of non-communicable diseases.
Quantitative indicator values have been normalised using distance to a reference country, that is, the country in which the best practice intervention is currently implemented (also referred to as the best practice “owner” setting) (OECD and European Commission, 2008[6]).
The normalisation equation is below:
Where:
= normalised value for target setting (country c) for indicator i
= original value for target setting (country c) for indicator i
= original value in the owner setting for indicator i.
Normalised values for equation (1) can be interpreted as percentage distance each country is from the best practice owner setting, whose value is centred on 0. Normalised values were used to allocate countries into one of five groups for each indicator, with a darker shade indicating greater transferability potential:
Value equal or greater than 0 = 
Value less than 0 but greater than ‑25% = 
(+25% when a lower value indicates better transferability)
Value less than ‑25% but greater than ‑50% = 
(>+25% but less than <+50%)
Value less than ‑50% but greater than ‑75% = 
(>+50% but less than <+75%)
Value less than ‑75% = 
(>75%)
For binary indicators, countries that respond “Yes” to the indicator are allocated the darkest shade (
) while countries that respond “No” are allocated the lightest shade (
).
For categorical indicators, any country that responds at least as well as the best practice owner are allocated the darkest shade (
), while the remaining countries are allocated a lighter shade based on the number of remaining categories.
OECD has developed a methodology to cluster countries and to make personalised recommendations on which member states and member countries are more likely to successfully transfer a recognised best practice intervention. A high level summary of the clustering methodology is below.
Cluster analysis partitions data into homogenous groups, based on similarities in the data. In this case it was used to separate countries into groups with similar characteristics, based on how well adapted or suited they are for transfer of a best practice intervention from a host country. For each cluster, specific recommendations can then be made to address potential obstacles for implementation. This can help guide decision makers and potentially lead to the smoother implementation and increased success of interventions.
To select the best methodology, 4 different cluster methods were compared: k-means, k-medoids, hierarchical and DBSCAN (Density-Based Spatial Clustering of Applications with Noise). K-medoids using Gower distance was found to be the most effective method for clustering countries taking into account validation statistics, data characteristics, interpretability of the results and flexibility to use with other datasets. This is because it works with small, imbalanced datasets with missing data, and can accommodate categorical data as well as continuous data.
The k-medoids algorithm is based on the medoid: this is the most central observation (country in this case) in the cluster, where the total distance between it and all the other countries in the cluster is smallest. Distance is a quantitative measure of dissimilarity, where the larger the distance between two observations, the more different they are from each other. The number of clusters (k) must be chosen prior to running the algorithm.
The k-medoids algorithm has the following steps:
Randomly assign k countries as medoids.
Repeat until there is no change in assignment of medoid:
Assign each country to a cluster, based on distance to the closest medoid.
For each cluster, test whether selecting another country as the medoid decreases the total distance from the medoid to all other points in the cluster. If it does, reassign this country as the new medoid.
Gower distance was chosen because it is able to compute the difference between both categorical and continuous variables. Gower distance is calculated from the mean of the partial pairwise distances between observations (countries). The partial pairwise distance is the difference between two observations at a single variable and is calculated differently depending on whether the variable is continuous or categorical.
Continuous Variables: The partial pairwise distance, between two observations and , for variable is the difference between the two values and , divided by the maximal range () of all the values for variable , as follows:
Categorical Variables: If two countries have the same value for a categorical variable then the partial pairwise distance is 0 (identical). Otherwise, it is 1.
The Gower distance between two observations is then calculated as the mean of the partial pairwise distances. The partial pairwise distances can be weighted differently. Here, the variables were weighted so that each contextual factor had equal weighting and therefore equal influence on the Gower distance. The resulting value lies between 0 and 1, with values closer to 0 indicating greater similarity between countries and values closer to 1 indicating greater dissimilarity. If one or both values are missing for a given variable in a pair of countries, the partial distance for that variable will not be included in the Gower distance, meaning there is no need for data imputation. However, if a country had over 50% variables missing it led to inaccurate Gower distances and so these countries were removed.
The clusters were compared by calculating the difference between the mean of each cluster and the mean of the dataset, for each indicator. A positive difference meant a higher likelihood of successful transfer for that indicator, allowing the characteristics of each cluster to be identified. To more broadly compare clusters, identifying the contextual factors (or domains) where clusters were stronger or weaker, domain scores were created and used to compare cluster means. Domain scores were created using the following steps:
Assign categorical variables dummy values (0 = no, 1 = yes).
Normalise using min-max scaling.
Aggregate by the mean of the variables in each contextual factor.
In summary, the following steps are required:
Remove countries where >50% variables are missing.
Compute a Gower Distance Matrix, with each contextual factor having equal weighting.
Determine optimal value of clusters (k) between 3 and 5.
Run k-medoids clustering using the optimal number of clusters from step 3.
Create domain scores in order to compare cluster means with the dataset means, and identify strength and weakness of each cluster.
Further details is available at Wiper et al. (2022[7]).
Limitations associated with the analysis of case study interventions are summarised in Table A A.3.
[4] Burchett, H., M. Umoquit and M. Dobrow (2011), How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks, https://doi.org/10.1258/jhsrp.2011.010124.
[1] European Commission (2021), Public Health Best Practice Portal, https://webgate.ec.europa.eu/dyna/bp-portal/ (accessed on 4 October 2021).
[2] König, H. et al. (2023), “Excess costs of mental disorders by level of severity”, Social psychiatry and psychiatric epidemiology, Vol. 58/6, pp. 973-985, https://doi.org/10.1007/S00127-022-02298-8.
[5] OECD/WHO Regional Office for Europe (2023), Mental Health Systems Capacity Questionnaire 2023.
[6] OECD and European Commission (2008), Handbook on constructing composite indicators: methodology and user guide, OECD (the Statistics Directorate and the Directorate for Science, Technology and Industry) and the Applied Statistics and Econometrics Unit of the Joint Research Centre (JRC) of the European Commission, https://www.oecd.org/sdd/42495745.pdf.
[3] Trompette, J. et al. (2014), “Stakeholders’ perceptions of transferability criteria for health promotion interventions: A case study”, BMC Public Health, Vol. 14/1, pp. 1-11, https://doi.org/10.1186/1471-2458-14-1134.
[7] Wiper, O. et al. (2022), “Cluster analysis to assess the transferability of public health interventions”, OECD Health Working Papers, No. 133, OECD Publishing, Paris, https://doi.org/10.1787/a5b1dcc1-en.