This chapter focuses on performance information, budget impact assessments and public policy evaluation as tools that help inform spending decisions and improve expenditure performance. Most OECD countries include performance information in budget reports and related documents. Many central budget authorities use impact assessments to better understand the potential impacts of budget measures, and public policy evaluations to provide essential information on whether budget measures were effective relative to policy objectives.
Quality Budget Institutions
Developments in OECD countries
Abstract
Informed spending decisions are based on an understanding of how budget proposals may influence outcomes that matter to the government. Shifting the focus of the budget process from inputs (“how much funding does the ministry receive?”) to measurable results (“what can be achieved with these resources?”) helps to achieve the effective and efficient use of government resources.
This chapter looks at the way OECD countries can use performance information, impact assessments and policy evaluations to inform budget decisions.
Performance budgeting creates a link between budget programmes and performance indicators to show what has been achieved from implementing the programmes. Performance budgeting increases the transparency and accountability of the government’s operations relative to its policy objectives. The performance information can also offer early indication of when programmes are not performing as expected.
Budget impact assessment is used to understand how budget proposals may impact on specific policy goals, both positively and negatively. Assessments can be applied to individual policy measures or to a portfolio of measures that relate to similar outcomes. Budget impact assessments are a key tool for gender and green budgeting.
Public policy evaluation considers how budget proposals have performance relative to the policy objectives they were designed to address. Ex post evaluation is the best-known form, and measures whether an intervention met its intended goals. Definitions of public policy evaluation include analysis at different stages of the policy cycle (i.e. during programme design or during implementation) and for a range of purposes (i.e. for learning or accountability).
These tools are becoming increasingly sophisticated. They are used alongside other practices such as spending reviews and performance audits to generate information on how to improve the impact of public spending and enhance value for money.
Performance budgeting has a long history in OECD countries, and CBAs have strengthened their use of it over time by increasing the quality and relevance of performance information, improving the accountability arrangements, strengthening the enabling environment and enhancing the presentation of performance information in budgetary documentation.
6.2.1. OECD country experiences with performance budgeting
Performance budgeting is defined as the systematic use of performance and results information to inform budget decisions, either as a direct input to budget allocation decisions or as contextual information to inform budget planning. Performance budgeting provides information on what ministries have done, or expect to do, with the resources allocated to them. This links the budget more clearly to government priorities and offers parliament and the public more information on the results of public spending.
There is no one-size-fits-all design to performance budgeting, but based on experiences from OECD countries, some factors can facilitate the introduction of performance budgeting. These include clear, transparent objectives and incentives for major stakeholders, and a practical design of the performance budgeting framework, as shown in Figure 6.1.
Source: Tryggvadottir and Bambalaite (2023[1]).
In 2023, a total of 28 out of 33 OECD countries reported that they have adopted performance-based budgeting. Each country has adapted the concept to suit its specific needs. Across the diverse practices, countries tend to use one of three approaches:
Presentational performance budgeting (7 countries) is where performance information is shown separately from the main budget document.
Performance-informed budgeting (14 countries) is where performance information is included in the budget document alongside financial information, with the aim of guiding budget allocations.
Managerial performance budgeting (7 countries) focuses on informing management decisions and organisational behaviour.
No OECD country directly links spending allocations to performance information, so that spending rises and falls in direct response to changes in results.
Share of OECD countries reporting different practices, 2023
Note: Data for Colombia, Ireland, Israel, Italy and Japan are not available.
Source: OECD (2023), Performance Budgeting Survey, Question 3.
Countries have reported a number of benefits of using performance budgeting, including more efficient allocation of funds and a better understanding of government priorities, as well as greater transparency and accountability throughout the budget process. It can also improve the dialogue between finance ministries, line ministries and parliament.
However, OECD countries also face challenges with performance budgeting (Figure 6.3). These include:
Lack of impact on budget decisions: One of the most persistent challenges is the limited use of performance information in shaping budget allocations. Even when performance information is available, budget decisions can continue to reflect historical spending patterns.
Information overload: An oversupply of data makes it difficult for the government to identify which information is most relevant for decision-making.
Limited embeddedness: Performance budgeting requires well-established processes and a supportive culture to use information for decision-making. Where this culture of performance is not established, performance budgeting can struggle to take root.
Capacity constraints: Performance budgeting requires administrative resources to generate and make use of performance data. Inadequate resources, and staff training and gaps in expertise are the most common issues raised by OECD countries.
Number of OECD countries reporting each challenge, 2023
Note: Data for Colombia, Ireland, Israel, Italy and Japan are not available. Category “other” is excluded from the analysis.
Source: OECD (2023), Performance Budgeting Survey, Question 25.
6.2.2. Key components of the effective performance budgeting framework
Relevant performance information
When introducing performance budgeting, OECD countries tended to include a large number of indicators in the budget documentation. However, as experience with performance budgeting has grown, CBAs have sought to identify fewer, but more relevant indicators to inform spending decisions and budget oversight. Estonia, France, Iceland, and the Netherlands have all reduced the number of performance indicators in budget documents.
In France, the first annual performance report in 2006 specified 650 goals and 1 300 indicators for 130 programmes in the general budget. In 2024, these were reduced to 441 goals and 882 indicators for 151 programmes.
In Iceland, significant efforts were made in 2017–2018 to improve the quality and focus of performance information. This involved introducing guidelines and limits on the number of objectives and indicators that each ministry could present.
The Netherlands halved the number of performance indicators in budget documents, reducing the total number by around 1 000 indicators.
Finance ministries often prepare guidelines and manuals to support line ministries in developing relevant performance information. Examples of such guidance include Australia’s Resource Management Guide on Developing Performance Measures and Greece’s Performance Budgeting Manual. These documents can include numeric limits on the number of objectives and indicators to be developed for each programme. In 2023, nine OECD countries included a limit on the number of indicators for developing performance information. Other countries may not have limits on the number of indicators for each ministry or programme, but review the relevance of the indicators in budget documents.
Accountability and transparency
Countries use performance budgeting to strengthen accountability and transparency of the results of spending decisions. Performance budgeting has typically had the greatest success when it responds to parliamentary requests for better information about the results achieved from the budget. This usually means developing reporting processes with the ministries and programme managers who use the performance information as it provides more meaningful information to parliament. To enhance accountability and policy relevance, many OECD countries require high-level sign-off of performance information, for example by a permanent secretary. However, to date OECD countries have rarely used sanctions, such as automatic budget cuts, when performance targets are not met. Instead, they encourage ministries to take ownership of the targets and make better use of information (Figure 6.4).
Number of OECD countries reporting each practice, 2023
Note: Data for Colombia, Ireland, Israel, Italy and Japan are not available. The category “other” is excluded from the analysis.
Source: OECD (2023), Performance Budgeting Survey, Question 11.
Advances in digital reporting have made it easier for governments to publish performance information. Previously, performance information was typically published on government websites in non-accessible formats (e.g. PDF). In recent years, there has been a shift towards more accessible and user-oriented approaches. In several OECD countries, performance information is now embedded into the websites allowing for more targeted communication with relevant stakeholder groups.
At the same time, many OECD countries have centralised online portals that consolidate budget and performance information. These portals are becoming more interactive, incorporating dashboards and data visualisation tools that facilitate user engagement and allow for deeper exploration of results. Examples of dashboards include Austria’s Wirkungsorientierte Datenwelt, Canada’s InfoBase, Estonia’s Budget Dashboard and France’s Performance de la Dépense dashboard, Mexico’s Budget Transparency Portal and the United States Data.org website.
Enabling environment
Two foundational elements that create the enabling environment for performance budgeting are programme budgeting and central co-ordination to build capacities.
Structuring the budget around the government’s key objectives and policy outcomes, often referred to as programme budgeting, offers several important benefits for budgeting and PFM, as it:
Enables better prioritisation of expenditure by clearly identifying policy goals across sectors and linking them to the resources allocated to achieve them.
Makes the budget more accessible and understandable to a broad range of stakeholders, including parliamentarians, civil society, and the general public.
Strengthens accountability within line ministries by showing what the ministry is responsible for relative to the government’s policy priorities.
Improves both the relevance and quality of the budget process by shifting budgetary discussions away from detailed line items and towards the outcomes and impacts of each programme – fostering more strategic, results-oriented dialogue between the CBA, line ministries, and decision-makers.
The CBA usually leads the development and maintenance of the performance budgeting framework. As part of this role, it provides line ministries with guidance and tools to help ensure consistent implementation. It is also responsible for ensuring that supporting IT systems are capable of effectively managing and integrating performance information.
When France introduced performance budgeting, the Ministry of Economy and Finance established a “PB school” to strengthen the technical skills of public managers. The PB school operated under the Ministry of Economy and Finance’s training centre and was accessible to officials from all ministries. As well as providing training was a platform for sharing experiences across ministries.
Presenting performance information in budget documentation
OECD countries take a variety of approaches to integrating performance information into the budget process. In many cases, performance information is incorporated into the annual budget documents. These documents are often structured around programmes, which serve as the basis for appropriations voted on by the legislature. Performance information is then developed for each programme by ministries and presented either within the main budget document or in accompanying annexes, as illustrated by the French example (Box 6.1). In countries, such as Canada, performance information is included in departmental plans and reports. These documents form part of the budget documentation and aligns planning, performance, and funding decisions.
It is important to be selective about the type of performance information presented in the budget. Tailoring the level of detail to the needs of different stakeholders enhances the usefulness of the data in decision-making. For instance, programme managers may require detailed, operational information to support day-to-day management, while parliamentarians benefit more from concise, strategic insights that inform budget deliberations. Striking the right balance takes time and requires ongoing dialogue with key stakeholders to ensure the information is both relevant and usable. It is important to be selective what type of performance information is presented in the budget. Tailoring the level of detail to different stakeholders supports more effective use of the data in decision-making. For example, programme managers require detailed information to support day-to-day decision-making, while parliamentarians need concise, strategic insights to inform budget decisions. Striking the right balance takes time and requires sustained dialogue with key stakeholders.
France has strong links between key performance indicators at the national level and the budget. France’s organic budget law (LOLF) classifies expenditures by “missions” that bring together related programmes and are associated with high-level policy objectives and performance indicators. The budget is structured around organic missions, programmes and actions (or activities). The 2024 budget had 47 missions, 187 programmes each with 5-15 activities. Performance information is set by programme, where performance objectives are linked to the budget structure and the implementation of performance objectives is measured by the performance indicators, as shown in Figure 6.5.
Recent reforms have focused on streamlining the indicators to improve the clarity of information to parliamentarians and the public.
Source: Ministry of Economy and Finance, France.
Budget impact assessments are one of the tools that can help inform how budget decisions impact on specific policy goals, both positively and negatively. Budget impact assessments are increasingly embedded in “strategic budget initiatives”, which focus on economic, social and environmental outcomes, and involve more than one ministry portfolio.
6.3.1. Emphasis on outcomes in the budget process
The majority of OECD countries focus on outcomes through “strategic budget initiatives” including gender budgeting, green budgeting and wellbeing budgeting (Nicol, 2023[2]). Countries have used the budget to progress outcomes in different ways, including through changes to performance budgeting which develop overarching programmes that involve more than one ministry portfolio, like the “missions” in France. However, the focus on outcomes is most visible in strategic budget initiatives such as gender budgeting and green budgeting. These initiatives help governments address key policy priorities systematically across ministry portfolios:
At their core, strategic budget initiatives aim to enhance decision-making by analysing the impact of budget measures on various social, economic, and environmental dimensions. They promote greater analytical rigor in policymaking and encourage evidence-based decision-making across the budget process. Additionally, they foster alignment between budgetary choices and overarching government priorities. (Nicol, 2023[2])
The first strategic budget initiatives were implemented by OECD countries in the 1980s and 1990s, but the use has gathered pace since the mid-2000s (Figure 6.6). By 2022, strategic budget initiatives had been established in 30 out of 38 nearly 80% of OECD countries. This rapid rise followed some general trends:
Gender budgeting paved the way for other strategic budget initiatives. It was first introduced in Australia in 1984, but was later adopted by other OECD countries, including Austria, Belgium, Finland, Iceland, Mexico, Norway and Spain between 2005 and 2009. By 2022, gender budgeting was being practised in 23 OECD countries.
Green budgeting followed with a focus on climate and environmental policies. Many countries have adopted it in response to both international commitments and in response to the damage caused by extreme weather events. The number of OECD countries practising green budgeting nearly doubled in two years from 14 countries in 2020 to 24 countries in 2022.
A number of OECD countries have established multi-dimensional frameworks. A few OECD countries use the Sustainable Development Goals in budgeting for this purpose. Others have developed well-being frameworks and apply them to budgeting, for example, Canada (Quality of Life Framework). Countries like Ireland (Irish Well-being Framework) and Italy (Equitable and Sustainable Wellbeing Framework) also track progress in wellbeing indicators to help guide policy decisions.
Countries are increasingly running multiple strategic budget initiatives. Out of 30 OECD countries that have employed at least one strategic budget initiative in 2022, 17 countries practised two or more, while 6 countries used three or more (Canada, Colombia, Finland, Mexico, New Zealand and Spain).
Note: Colours represent the year a country first adopted a new practice.
Source: Nicol (2023[2]), updated by OECD Authors.
Strategic budget initiatives are influencing policy and spending decisions. In 2022, nearly half of OECD countries with gender budgeting reported that this was having an impact on policy development and resource allocation, either across government or in specific areas (OECD, 2023[3]). Over two thirds reported that green budgeting was being used to inform government decisions (OECD, 2024[4]).
However, OECD countries also face challenges implementing strategic budget initiatives of this nature. The most common challenges reported by countries are capacity constraints, in the form of insufficient resources or a lack of knowledge or expertise. Tools need to focus on measuring impact and should support decisions on key trade-offs and activities. Strategic budget initiatives also require effective co-ordination. This is particularly true for countries that are running more than one strategic budget initiative, where the proliferation of budget appraisal tools can add complexity to the budget process.
6.3.2. Combining tools and information for measuring impact
Strategic budget initiatives combine multiple tools to help the government assess of how the budget will impact policy outcomes. Table 6.1 lists some of the well-used tools for gender and green budgeting. Three approaches are applied relatively widely in OECD countries:
Incorporating goals and indicators in the performance budgeting framework. Over half (12 out of 23) of OECD countries using gender budgeting include a gender perspective in performance frameworks. Only 11 out of 24 OECD countries implementing green budgeting used a green perspective in performance setting in 2022, though both green and gender are often included in the multi-dimensional frameworks for wellbeing budgeting.
Assessing the impact of budget policies on specific goals. Gender impact assessments are conducted in 13 countries, either ex ante before the policy is implemented or ex post to assess whether policies have had the desired effect. A smaller set of countries are estimating the impact of budget proposals on carbon emissions to assists with forecast, and modelling the potential impact of the budget relative to carbon reduction plans and commitments.
Incorporating a gender or green dimension in spending reviews, public policy evaluations and performance audits. These tools can be used to assess policies specifically aimed at gender or environmental objectives, and to evaluate how policies with broader goals affect gender equality or environmental outcomes.
Note: Share out of 23 countries implementing gender budgeting; share out of 24 countries implementing green budgeting.
Source: OECD (2022), Survey on Green Budgeting with responses from 36 OECD countries; OECD (2022), Survey on Gender Budgeting with responses from 38 OECD countries.
Countries are still learning which tools work best for running a successful strategic budget initiative, and how this varies across different policy areas. Not all budget tools will perform as expected. While this can draw criticism, it can also lead to innovations that improve existing tools or establish new approaches.
Even in countries with advanced green budgeting practices, for example, there are perceptions that the current tools are not sufficient for informing budget decisions (OECD, 2024[5]). Green budget tagging has been used widely to offer an overview of climate-related expenditures. Tagging can also distinguish qualitatively between expenditures that have favourable, mixed, neutral and unfavourable impacts on the climate and may extend to other environmental policies. However, tagging provides limited information on how spending materially impacts on emissions or the government’s climate goals. The tagging process can also lead to over-estimating the green credentials of a budget (European Court of Auditors, 2022[6]).
The next generation of green budgeting instruments aim to deepen the understanding of how spending decisions impact on environmental and economic outcomes. Denmark, for example, has strengthened macroeconomic modelling to support impact analysis. This modelling helps to understand how climate change will impact on the economy and how climate policies will impact on the government’s environmental and economic goals. The most sophisticated models allow policymakers to compare the economic, fiscal, social and climate impacts of different policies (OECD, 2024[5]).
The tools of budget impact analysis are being refined under other strategic budget initiatives in order to provide information that is relevant for decision making. Gender budgeting has explored the use of intersectional analysis that aims to look at how different forms of social inequality interact and if they are driven by common factors. Distributional analysis of taxes and cash benefits has expanded to include information on the use of in-kind services and to consider the distributional effects of tax and benefits over a person’s lifetime.
OECD countries often adapt approaches for strategic budget initiatives to different policy goals. Understanding economic, environmental or social goals requires specialist analysis and may also involve different stakeholders. However, multiple, or separate strategic budget initiatives may overload the budget process and undermine the benefits of prioritising important cross-sectoral goals. Several OECD countries have looked for ways to effectively combine different strategic budget initiatives. Canada, for example, prepares a Budget Impacts Report which contains information on the estimated impact of budget initiatives across multiple frameworks (Box 6.2).
Canada’s Statement and Impacts Report on Gender, Diversity and Inclusion meets the obligations set out in the Gender Budgeting Act (2018) which requires the Minister of Finance submit a report in parliament on the impacts in terms of gender and diversity of all new budget measures described in the annual federal budget. The report also includes analysis from the Quality of Life Framework, which includes climate and environmental assessments:
Gender-based Analysis Plus (GBA Plus) is an analytical tool used to support the development of responsive and inclusive policies, programmes, and other initiatives. It is an intersectional analysis that goes beyond biological (sex) and socio-cultural (gender) differences to consider other factors, such as age, disability, education, ethnicity, economic status, geography (including rurality), language, race, religion, and sexual orientation.
In 2021, the federal government introduced its Quality of Life Framework which tracks well-being and aims at supporting inclusive and sustainable growth. The Framework is composed of five domains – environment, good governance, health, prosperity, and society, and contains 84 indicators to help identify trends and changes in the quality of life for Canadians.
In 2024, the government strengthened its assessment of the climate and environmental considerations from budget measures by introducing a Strategic Environmental and Economic Assessment. The Assessment builds upon previous initiatives to include climate and environment assessments in the Budget Impacts Report.
Source: Department of Finance, Canada.
6.3.3. Accountability and transparency
Budget impact assessments help align spending proposals with high-level policy priorities. Budget impact analysis is more likely to help progress government goals if it can combine the assessments of individual proposals with an overview of how the proposals collectively contribute to the policy priorities in terms of costs and impact. A number of OECD countries, including France, Sweden and the Netherlands, have successfully integrated distributional assessments of tax and spending into the budget decision-making processes to guide decisions over income redistribution and labour market policy (Box 6.3). In these countries, distributional analysis is updated during budget formulation to inform decision-making and the final assessment is published with the government’s budget proposal.
Publishing the results of impact assessments reinforces accountability for budget decisions. Several countries publish budget statements, reports or annexes with the budget documents which provide detailed assessments of budget decisions on specific outcomes. In Australia the Department of the Treasury prepares a Women’s Budget Statement as part of the budget papers. The Statement includes statistics and a qualitative assessment of gender equality in Australia as well as funding commitments and corresponding policy announcements. In New Zealand, the government amended the Public Finance Act to publish an annual report child poverty on the same day as the budget. Spain publishes “cross-cutting” reports that cover different themes each year (e.g. on child, adolescent and family impact in 2021). Canada and Ireland publish reports that summarise the impact of budget measures on different cross-sector policies.
Distributional analysis is usually focused on taxes and cash transfers, though some countries also include in-kind services such as access to state-funded education and health care. Together, these policies have a significant influence on the distribution of income across society. They also shape people’s behaviour in the labour market. Distributional analysis helps to ensure that the budget is consistent with the government’s goals for income redistribution and for labour market policy.
Table 6.2 shows that distributional analysis is used in many countries to prepare the budget ex ante and to review its impact ex post. Governments prepare their own analysis, but this is often complemented by similar work in academic institutions or independent fiscal institutions. Most countries include the analysis in the budget documents, which provides a basis for external accountability.
6.3.4. Enabling environment
Strategic budget initiatives typically focus on outcomes that require cross-governmental responses. Countries are most likely to make progress if they can establish clear objectives and a stable policy framework. Governments should ideally have strategic budget initiatives linked to goals set out in a national or sectoral strategy. In 2022, 20 out of 24 OECD countries had a national climate or environmental strategy to inform green budgeting (OECD, 2024[4]). Similarly, gender budgeting is guided by national strategies and sectoral goals. The legal framework for strategic budget initiatives are still being developed. However, some countries have created requirements in law; 14 out of 23 OECD countries that practice gender budgeting have a legal requirement to do so.
To improve social, economic or environmental outcomes, the government needs to be able to co-ordinate spending decisions across portfolios. These initiatives are most successful when the CBA has a strong leadership role. More than half (61% in 2022) of CBAs in OECD countries perform a key role ingender budgeting, for example by:
Managing the framework, tools and guidelines used to guide budget decisions.
Providing advice and quality assurance for analysis prepared by line ministries.
Identifying and addressing gaps in the capacity of line ministries and agencies.
However, the CBA will often have shared responsibilities with another ministry or agency that has deep expertise in the policy area of the initiative. This was the case for gender budgeting in Austria, Israel, Lithuania, New Zealand and Norway. Similarly, for green budgeting, inter-departmental committees help to provide co-ordinated responses by government to budget initiatives.
Budget impact analysis depends on the availability of good quality, disaggregated data. Countries with experience of gender budgeting are reporting improvements in the availability of disaggregated data. The share of countries reporting that disaggregated data was available for all or most areas of the public service rose from just 16% in 2016 to 61% in 2022. Ireland is among the countries that has made progress with the availability of data (Box 6.4). Alongside the finance ministry and line ministries, the national statistics office performs a crucial role in providing the data and maintaining the data standards for the indicators used in strategic budget initiatives, including in well-being budgeting frameworks as demonstrated in Canada.
In 2020, the Central Statistics Office (CSO) of Ireland conducted a data audit of 107 data sources from 31 public bodies in co-operation with the Department of Public Expenditure and Reform. The audit aimed to ascertain the availability of public service data disaggregated by equality. In 2021, a focused equality data audit was conducted, covering all national data sources held by Tusla (the Child and Family Agency). This work was guided by the Equality Budgeting Expert Advisory Group representing key internal and external stakeholders and the audit findings were published alongside Ireland’s 2021 Budget. The information is also published on the CSO webpage and will continue to be updated as new data is identified.
In response to the data audit, the CSO and the Department of Children, Equality, Disability, Integration and Youth are developing a data strategy to identify what actions are needed to improve the disaggregation of data and identify actions needed to address data gaps.
Source: OECD (2022), Survey on Gender Budgeting.
‘Public policy evaluation’ refers to the structured and evidence-based assessment of the design, implementation or results of a planned, ongoing or completed public intervention. It assesses the relevance, coherence, efficiency, effectiveness, impact and/or sustainability of a policy based on its objectives (OECD, 2022[8]).
Public policy evaluation provides an objective assessment of what works, why, for whom, and under what circumstances. Evaluation covers a wide (and growing) array of methodologies. They can be conducted at different stages of the policy cycle: ex ante to inform the design of a new policy, ex durante to guide and monitor implementation or ex post to understand whether an intervention has had the desired impact. They may also be used for different purposes, including for learning or accountability.
Policy evaluation is a key tool for making informed policy and spending decisions, and can also enhance the transparency and accountability of government actions. Most governments in OECD countries have established laws, tools and organisational arrangements for a cross-governmental approach to public policy evaluation. Yet, despite these instruments for systematising policy evaluation, countries also raise concerns that evaluations have insufficient coverage in the public sector, suffer from variable quality and can have limited influence over spending decisions.
To strengthen evaluation capacities, OECD countries are focusing reforms on institutionalising evaluation through a whole of government approach, promoting the quality of public policy evaluations and ensuring that evaluations impact decision making, through the policymaking and budget process (OECD, 2022[8]).
6.4.1. Institutionalising from a whole of government perspective
Institutionalising policy evaluation from a whole of government means embedding public policy evaluation across public entities. It ensures evaluations are carried out in a systematic manner with routines that foster a culture of learning and accountability in the executive and beyond.
Planning effectively
Countries should actively plan, design, and manage evaluations so that they are timely and proportionate, taking into account the needs of the primary users. Policy frameworks can guide line ministries to establish plans for what will be evaluated and how. Some countries like Japan and Switzerland have adopted a comprehensive approach to evaluation. Others deploy evaluations more selectively, requiring line ministries to set out plans that systematically identify which policies should undergo an evaluation, as is the case in Italy and the United States. Canada has used both approaches, with the Policy on Results prompting a transition from the more comprehensive to more selective approach.
Central co-ordination and evaluation champions
A whole of government approach typically combines central co-ordination with greater capacity in line ministries and other public entities to evaluate government programmes. Co-ordination involves managing the policy framework for evaluation, developing standards, sharing best practices and providing support to public entities and evaluation officers across government. This is often a responsibility for the centre of government or the ministry of finance (and in several cases a shared one), but may also involve other bodies such as a ministry of planning.
The centre of government usually provides strategic direction for policy evaluation. It usually leads the evaluation function across government and advises ministries on best practices. This fits closely with the centre of government’s role co-ordinating policy for the government.
The finance ministry provides a more direct link to the budget process. Finance ministries often manage the cross-government policy evaluation programme and lead on cross-sectoral policy evaluations. In 2023, nearly half of finance ministries reported that they have a dedicated evaluation unit that support these responsibilities.
Many OECD countries have created roles to “champion” the use of evaluation in government, including by co-ordinating cross-government policy evaluation, sharing best practices and encouraging follow-up on evaluation findings. Examples the Irish Government Economic Evaluation Service (IGEES) in Ireland, the Strategy Analysis Unit of the Ministry of Finance in the Netherlands, and Portugal’s PlanApp.
Assigning responsibilities for conducting evaluations systematically
Line ministries are typically responsible for commissioning and conducting policy evaluations of their own policies and programmes. They are also expected to act on findings and feed results into future policy cycles. Dedicated evaluation units in line ministries are helping to strengthen the planning, implementation and monitoring of evaluation, though the coverage of these units varies. In 2023, a majority of OECD countries (22 out of 31, 71%) have dedicated evaluation units in some or most line ministries but only Canada, Mexico, the Netherlands and Spain report having dedicated units in all line ministries (OECD, 2023[9]).
Building a learning and accountability culture
A successful evaluation system support a culture of learning and accountability. This typically goes beyond the use of evaluation alone, to include other tools for performance management and evidence-informed policy making, as was the experience in the United States with the 2018 Evidence-Based Policymaking Act. The first phase of the reform involved developing learning agenda, identifying relevant personnel, their roles and responsibilities, and undertaking planning activities (Box 6.5).
Australia
The Australian Centre for Evaluation (ACE) was established within the Department of the Treasury in July 2023 as the institution responsible for promoting the systematisation, quality, and use of evaluation and evidence across the Australian Public Service (APS). The Centre has been allocated a budget of AUD 10 million over a four-year period. Its primary objectives are to support the implementation of the Commonwealth Evaluation Policy, adopted in 2021; to oversee the overall quality of evaluations conducted by government departments and agencies; and to assist these entities in undertaking evaluations using robust methodologies, including randomised controlled trials.
Japan
The Japanese government embedded evaluation across its administration as early as 2001 with the Government Policy Evaluations Act No. 86. The Act provides a framework to assess the impact of proposed legislation, incorporating results into planning, and defines the roles of administrative organs responsible for these assessments. It mandates guidelines on ex ante regulatory impact assessment and mechanisms for reporting results to the legislative body (the Diet) and the public. In 2017, the guidelines were further updated to include ex post evaluation.
United States
In the United States the 2018 Foundations for Evidence-Based Policymaking Act provides a legislative and bipartisan approach to policy evaluation in the federal government. The Office of Budget and Management oversees the implementation of the Act and supports it with guidance materials. The Act defines and assigns institutional responsibilities for conducting policy evaluations. These responsibilities include designated positions, referred to as Evaluation Officers, to co-ordinate evaluations and advise on evaluation priorities.
Sources: Department of the Treasury, Australia, Ministry of Finance, Japan, Office of Management and Budget, United States.
6.4.2. Promoting quality evaluations
Quality evaluations require timely and rigorous processes and methodologies. Promoting high-quality evaluations is crucial to ensure they are trusted and used for decision-making, learning, and accountability. In contrast, low-quality evaluations may result in unreliable or biased findings, reducing their usefulness for making more informed spending decisions.
Countries should establish standards for policy evaluations to generate robust and credible evaluation results that can be trusted and used with confidence. This means using robust methods and quality assurance.
Australia: The Evaluation Policy and Toolkit provides an example of guidance on how to plan and conduct evaluations (Australian Centre for Evaluation, 2023[10]).
Ireland: The Public Spending Code set outs the government’s expectations on policy evaluation and the Department of Public Expenditure and National Development Plan Delivery and Reform is responsible for approving any sector-specific variations to those standards.
The United Kingdom: The central government guidance on evaluation (Magenta Book) and the guidance on appraisal and evaluation (Green Book) provide detailed guidance that applies across all central government ministries and agencies.
The 2023 OECD Survey on Public Policy Evaluation (OECD, 2023[9]) shows that OECD countries have room to improve practices that help make evaluations more objective. Outsourcing remains the go-to approach for ensuring the independence of evaluations in OECD countries (used in 73% of countries, 23 out of 31 OECD countries). Fewer than half of countries have a peer review system (58% have no peer review system, 18 out of 31 OECD countries) and where these are used, they apply to a subset of evaluations (38% of countries, 12 out of 31 OECD countries). Only 9 out of 31 OECD countries (29%) have a pre-defined quality criteria .
As with other tools for informed decision-making, countries often need to develop and refine their capacity to commission and conduct evaluations effectively. OECD countries mostly rely on regular training (15 out of 31 OECD countries, 48%) while a few have created special cadres in the public service (4 out of 31, 13%). Ireland is an example of a country that has taken a systematic approach to building capacity for economics analysis, statistics and evaluation across all line ministries (Box 6.6).
The Irish Government Economic and Evaluation Service (IGEES) applies EIPM across the whole of government. IGEES has contributed to building capacities in all government departments in the areas of economics, statistics, and policy evaluation. It aims to improve the design and performance of policies and contribute to better outcomes for citizens. Key components IGEES work are from conducting spending reviews and publishing reports that apply evaluation, value for money analyses, and other methodologies to consider whether government policies have performed as intended. IGEES reports have considered the drivers of pension reform, management of healthcare expenditure, social impact analysis and many other topics. IGEES staff are integrated in each department to provide a cross-governmental network of expertise and support. The network is led and co-ordinated by the Department of Public Expenditure, National Development Plan Delivery, and Reform. The IGEES staff are either existing employees who have rotated into IGEES roles or staff directly recruited through an open and competitive process.
Source: Department of Public Expenditure, National Development Plan Delivery, and Reform, Ireland.
6.4.3. Impacting budget decisions
OECD countries report that evaluations are more likely to be used for programme implementation or strategy development than for budget decisions. This is at least partly because many ex post evaluations are used to inform government decisions on whether to continue an existing policy or programme, or to make changes to improve it.
This means that policies and budgets may benefit from processes that bridge policy evaluation processes and budgeting processes. Experiences in OECD countries shows that they tend to follow some common strategies. One is to integrate the findings from evaluations into other tools that are more directly connected to budget decisions, such as spending reviews or agency performance reports. Another is to provide easy access to evaluation results and make them more relevant to spending decisions.
A number of OECD countries have developed ways to synthesise and link the findings from policy evaluations to the main routines for budget preparation and oversight.
Requirements to explain evaluation plans and findings: In Canada, submissions for new spending proposals to the Treasury Board must specify the expected results of the policy, whether the design has been informed by an evaluation and if there are future evaluations planned. These elements of the proposal must be verified by the Deputy Head (who is responsible for evaluation and performance measurement (Treasury Board of Canada, 2022[11]).
Using evaluation findings to prepare for budget negotiations: In Lithuania, the Office of Government together with the Ministry of Finance summarises the results of evaluations in preparation for budget negotiations. They prepare a note with information on progress achieved by the relevant line ministry since the evaluation and any implementation gaps (OECD, 2020[12]).
Using evaluation findings to inform regular spending reviews: In the Netherlands, and in other countries, spending reviews often synthesise the results of relevant public policy evaluations (de Jong, 2022[13]). The 2023 OECD Survey on Spending Reviews found that public policy evaluation was the most commonly used source of information for conducting spending reviews (reported in 22 out of 33 OECD countries).
Making evaluation findings available to the public and in more accessible formats can help increase their use and impact. The first essential step is to make sure that evaluations are public by default, this occurs in 61% of OECD countries (19 out of 31) that responded to the survey (OECD, 2023[9]). However, this is not enough to make sure that results will be visible and digestible to different audiences. For this reason, it is also important to develop user-friendly tools to summarise evaluation results. In Canada, Departmental Performance Reports are published by individual departments and agencies. They detail accomplishments, results achieved against performance targets, and may include summaries of evaluations of the department's programmes. France, Ireland, Lithuania, Norway, and Portugal are a few of the growing list of countries that have created online portals for public policy evaluations, providing general access to these studies (Box 6.7).
Norway’s online evaluation portal is a publicly accessible web service that gathers the findings of evaluations carried out by the central government. The database is operated by the Directorate for Financial Management and the National Library of Norway.
It contains evaluations carried out on behalf of government agencies from 2005 until today, as well as a selection of central evaluations from 1994 to 2004. Evaluation reports are registered in the database as soon as they are available to the public. Moreover, the portal provides evaluation guidelines, a calendar of the key activities in the evaluation area, news and professional papers. By increasing accessibility to evaluation results, the portal allows the use and reuse of the knowledge and findings from evaluations in all state policy areas, in future evaluations and in society as a whole.
Source: OECD (2020[12]).
References
[10] Australian Centre for Evaluation (2023), Commonwealth Evaluation Toolkit, https://evaluation.treasury.gov.au/index.php/toolkit/commonwealth-evaluation-toolkit.
[13] de Jong, M. (2022), Spending review practices in the Netherlands, PEMPAL, https://www.pempal.org/knowledge-product/spending-review-practices-netherlands (accessed on 30 March 2025).
[6] European Court of Auditors (2022), Climate spending in the 2014-2020 EU Budget not as high as reported, https://www.eca.europa.eu/lists/ecadocuments/sr22_09/sr_climate-mainstreaming_en.pdf.
[2] Nicol, S. (2023), The emergence of strategic budget initiatives: some preliminary observations, OECD, Paris, https://one.oecd.org/document/GOV/SBO(2023)21/en/pdf (accessed on 11 March 2025).
[5] OECD (2024), “Beyond green tagging: How can public budgeting support climate goals?”, OECD Papers on Budgeting, No. 2024/06, OECD Publishing, Paris, https://doi.org/10.1787/b154d49b-en.
[4] OECD (2024), Green Budgeting in OECD Countries 2024, OECD Publishing, Paris, https://doi.org/10.1787/9aea61f0-en.
[3] OECD (2023), Gender Budgeting in OECD Countries 2023, OECD Publishing, Paris, https://doi.org/10.1787/647d546b-en.
[9] OECD (2023), Survey on Public Policy Evaluation, https://data-explorer.oecd.org/.
[8] OECD (2022), “Recommendation of the Council on Public Policy Evaluation”, OECD Legal Instruments, OECD/LEGAL/0478, OECD, Paris, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0478.
[12] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/89b1577d-en.
[7] OECD/KIPF (2024), Addressing Inequality in Budgeting: Lessons from Recent Country Experience, OECD Publishing, Paris, https://doi.org/10.1787/ea80d61d-en.
[11] Treasury Board of Canada (2022), Treasury Board Submissions.
[1] Tryggvadottir, Á. and I. Bambalaite (2023), “OECD Performance Budgeting Framework”, OECD Journal on Budgeting, Vol. 2023/3, https://www.oecd.org/en/publications/oecd-journal-on-budgeting-volume-2023-issue-3_ec1d2b4e-en.html.