Government at a Glance 2025

Annex C. Methodology for the fiscal advocacy index

The 2024 OECD Fiscal Advocacy Index is a tool for assessing the extent to which independent fiscal institutions (IFIs) fulfil the role of fiscal advocates — institutions that champion fiscal sustainability.

The Index evaluates IFIs across four dimensions:

1. Independence - the extent to which IFIs effectively provide impartial advice to policymakers and the public. Leadership and operations should be free from outside interference and funding should be multiannual in nature, preferably with direct approval of funding from the legislature. This dimension is based on an update of earlier OECD work measuring IFI independence (Nicol and Von Trapp, 2018[1]).

2. Analysis - as fiscal advocates, IFIs should be able to focus their work on major aspects of fiscal policy. IFIs should be able to explore long-term sustainability challenges and major fiscal risks facing the public finances. Those that focus on costings should be able to promote citizens’ interests by bringing greater transparency to government or legislative proposals as well as election platforms. At the same time, they should have the capacity to generate and deliver macroeconomic and fiscal forecasts to support their analysis, with an adequate number of analytical staff.

3. Communications Apparatus - the ability of an IFI to communicate directly with the public and engage with the media separates it from other non-partisan research services. To do this effectively, IFIs should have robust tools in place to disseminate, promote and track the impact of their work. This dimension incorporates previous OECD work, the OECD IFI Communications Index (OECD, 2023[2]).

4. Communications Impact - having a robust communications function can help generate an impact on the national debate and promote the IFIs analysis. However, it does not guarantee it. The impact of an IFI’s work often relies on more than just the methods it uses to promote that work. It relies on clear and cogent analysis in key areas. This dimension gauges the final impact an IFI has by considering its prevalence in media discussions and the public’s interest in the institution.

The four dimensions have been informed by the OECD Council Recommendation on Principles for Independent Fiscal Institutions (OECD, 2014[3]). The Recommendation contains 22 principles across nine areas related to local ownership, independence and on partisanship, scope of the mandate, resources, relation with the legislature, access to information, transparency, communication and evaluation.

The methodology used for building the 2024 OECD Fiscal Advocacy Index is based on the Handbook on Constructing Composite Indicators (OECD/European Union/EC-JRC, 2008[4]). The index has also been shared and discussed with delegates from the OECD’s Working Party of Parliamentary Budget Officials and Independent Fiscal Institutions

Data collection and validation

For the first three dimensions, data used for the construction of the 2024 OECD Fiscal Advocacy Index comes primarily from the OECD Independent Fiscal Institutions Database (OECD, 2021[5]). The data was collected via desk research and then verified and validated by relevant senior officials in the OECD’s Working Party of Parliamentary Budget Officials and Independent Fiscal Institutions.

In addition to that, the fourth dimension (communications impact) draws on new indicators, created using a combination of data on national media coverage plus Google Trends data for 2021 to 2023. These are intended to assess the extent to which IFIs have entered the national debate, as recorded in mentions of their work in the media.

Dimensions and weights

Each of the four dimensions is equally weighted, has a maximum value of 1, and comprises several variables. The overall index has a maximum value of 4, with institutional scores being the sum of the scores for each dimension. A higher score indicates a greater capacity for fiscal advocacy.

The variables and weights comprised in the index were selected based on their relevance to the concept by a group of experts within the OECD and in consultation with delegates to the Working Party of Parliamentary Budget Officials and Independent Fiscal Institutions. Variables within a dimension are weighted based on 1) the number of indicators making up each variable, and 2) and the relative importance of each variable.

A linear aggregation method is applied to first aggregate the indicators into variables, the variables into sub-dimensions and then the sub-dimensions into an overall dimension or composite indicator. To obtain overall index scores, the weighted scores for each indicator are totaled.

The components used in the construction of this index, and the weights given to each, are indicated in the figure below.

Figure ‎C.1. 2024 OECD Fiscal Advocacy Index: Dimensions, sub-dimensions and their weights

A detailed explanation on the components of the 2024 OECD Fiscal Advocacy Index will be available in a forthcoming publication (OECD, forthcoming[6]), including the variables, answer options, scores and weights used to construct the composite index, as well as the statistical analysis carried out.

Statistical analyses

Several statistical tests were used to show how reliable the index is in terms of measuring a coherent underlying concept – fiscal advocacy. They also assess how valid the choices of individual parameters and variables are.

Sensitivity analysis was carried out to establish the robustness of the index scores to different weighting options through Monte Carlo simulations. The results from the sensitivity analysis at dimension level for the 2024 OECD Fiscal Advocacy Index show that, for the majority of the institutions analysed, the overall scores are not very sensitive to the choice of weights given to the categories. In terms of the overall Index, the Cronbach’s alpha coefficient is equal to 0.89, indicating that the dimensions are measuring the same underlying construct.

References

[1] Debrun, R. (ed.) (2018), Measuring IFI independence: A first pass using the OECD IFI, CEPR Press.

[2] OECD (2023), Government at a Glance 2023, OECD Publishing, Paris, https://doi.org/10.1787/3d5c5d31-en.

[5] OECD (2021), Independent Fiscal Institutions Database (Version 2.0), OECD, Paris, https://www.oecd.org/gov/budgeting/OECD-Independent-Fiscal-Institutions-Database.xlsx.

[3] OECD (2014), “Recommendation of the Council on Principles for Independent Fiscal Institutions”, OECD Legal Instruments, OECD, Paris, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0401.

[6] OECD (forthcoming), From fiscal watchdogs to fiscal advocates: Creating champions of fiscal sustainability, OECD Publishing, Paris.

[4] OECD/European Union/EC-JRC (2008), Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Publishing, Paris, https://doi.org/10.1787/9789264043466-en.