Programme and performance budgeting in Chile

Annex A. Assessment against performance budgeting and policy evaluation frameworks

Table ‎A.1. Performance budgeting framework in Chile

Tools and methods for developing performance	Sequencing performance information	Moderate	Performance objectives are linked to the budget structure but not in a clear way Performance information includes performance goals, indicators and targets There are no multi-annual performance targets, 51% of OECD countries set those.
	Avoiding overload of performance information	Low	There number of performance information is expected to be stable overtime. With respect to 2019 the number of budget programmes has increased as well as the number of performance indicators. On average, both numbers have remained stable for most OECD countries. There is no formal limit on the number of performance objectives or performance indicators to be included for each programme. A limit is present in 27% of OECD countries
	Criteria for developing performance information	Moderate	There is no explicit rule to align performance information to government priorities (48% of OECD countries have them). Performance information is required to be outcome oriented and follow a SMART logic, in line with most OECD countries.
	Quality assurance process	Low	The Budget Office does not perform a quality assurance of the performance information included in the budget; this occurs in 61% of OECD countries.
Accountability and transparency	Managerial flexibility and accountability	Moderate	The Ministry of Finance engages in discussions with line ministries on the quality of performance information and its relevance to budgetary decisions, as in most OECD countries (22 out of 33). There are also monitoring and reporting mechanisms to track results, in line with most OECD countries (20 out of 33). There is no sign-off by a high-level official on performance information to be included in budget documentation, while this occurs in 36% of OECD countries. There are consequences for line ministries if the targets are not met however, these are not clearly specified. Across OECD countries, the most common consequence when results are not achieved is to increase monitoring of the organisation in the future (7 out of 33).
	Reporting practices	High	Line ministries are required to report on the intended targets and the actual results achieved in the year-end reports The end-year performance report includes both financial and performance information
	Roles and responsibilities	High	The Ministry of Finance sets the overall framework Line ministries are responsible for operational implementation of performance budgeting, including developing performance information, reporting and budget execution Supreme audit institution assesses the soundness of the overall framework and conducts performance audits within relevant areas Parliament uses the information to hold ministries accountable and monitor the implementation of performance targets but has an overall limited role.
	Public access to performance information	High	All performance information included in budget documentation is included in relevant performance reports Year-end performance report is published Performance information is published but not in an interactive dashboard
An enabling environment	Budget structured around programmes	Low	The budget has a programme structure; however programmes do not reflect specific objectives
	Centrally issued guidelines and templates	High	There are centrally issued guidelines or manuals, as in 89% of OECD countries. Standard templates for collecting and reporting on performance information are prepared by DIPRES, this is what performed by 70% of OECD countries.
	Capacity-building efforts	High	Trainings are provided to build capacity like in 56% of OECD countries.
	Supporting IT environment	Moderate	Performance information is collected through the same IT system, like in 74% of OECD countries. However, performance information and financial information are collected through different IT systems. 57% of OECD countries have integrated IT systems to collect both performance and financial information.
	Incentive mechanisms	Low	No formal incentives are put in place, only a minority of OECD countries have specific incentives in place.
Use of performance information to inform decisions around budget	Role of the Parliament	Moderate	Parliament monitors the implementation of performance targets included in the budget and uses performance information for accountability However, parliament does not have discussions with stakeholders to improve the performance information provided in the budget, while this occurs in 40% of OECD countries.
	Role of the sectoral committees in the Parliament	Low	Sectoral parliamentary committees do not engage with relevant line ministries regarding the performance information included in the budget
	Presentation of relevant information in budget documents	High	Performance information is presented in standardised manner both in the main body and in the annexes

Table ‎A.2. Policy Evaluation system in Chile

Institutionalisation of policy evaluation	Presence of evaluation champions	At standard	Chile has defined two evaluation champions with the mandate to co-ordinate evaluations across line ministries: DIPRES and the Ministry of Social Development The role and responsibilities of champions are clearly defined, and there is a clear distinction regarding those of other government institutions This is in line with most OECD countries, 84% of OECD respondents have at least one evaluation champion In most OECD countries, evaluation champions are in the Ministry of Finance (45%) or in the Centre of Government (39%)
	Institutional responsibility definition	In development	In Chile, dedicated evaluation units are presented in a minority of line ministries (below 50%), and these are developed at the discretion of ministries This is the case in almost 50% of the OECD countries, while in 12% of countries there are dedicated evaluation units in each line ministry (Canada, Netherlands, Mexico and Spain).
	Knowledge broker promotion	In development	Limited role of actors outside the executive in promoting evaluation (e.g., supreme audit office not involved). Interesting role of the CNEP in developing external evaluations. There are no specific knowledge brokers active in evaluation, as in the 50% of OECD countries. Chile also does not have a specific Evaluation Society, contrary to 70% of OECD countries.
	Legislative review opportunity	In development	The legislative body in Chile can require government to evaluate specific policies and present and discuss evaluation results. This occurs in majority of OECD countries. However, the legislative body does not have internal analytical functions to conduct evaluations. In 35% of OECD countries the legislative body has established an internal body responsible for conducting evaluations.
Mechanisms to ensure quality of policy evaluations	Early evaluation planning and provision	Opportunity for improvement	Evaluation clauses are included in some laws (below 50%). This occurs in 65% of OECD countries. There are no central requirements establishing when a policy needs to be evaluated. In other countries policies that results from international commitment need to be evaluated (51%) or policies that exceed certain budgetary thresholds (19%).
	Stakeholder engagement	Opportunity for improvement	There are no mechanisms to ensure that external stakeholders are consulted in the evaluation processes. In 70% of OECD countries, mechanisms are in place to ensure stakeholder engagement for at least some evaluations. In 22% of OECD countries, stakeholders need to be engaged in all evaluations.
	Guidelines on policy evaluation	At standard	In Chile there are central guidelines to support the evaluation process. Guidelines are present in 80% of OECD countries.
	Professional Standards	At standard	Chile has a public register of evaluation experts that requires specific professional standards for evaluators. This represents a best practice as 77% of OECD countries have no professional standards or accreditations systems in place. In addition, Chile has specific ethical standards in place to ensure the independence of evaluators. Most countries (80%) simply rely on ethical standards for all public servants.
	Autonomy of evaluations	At standard	Different mechanisms are in place to ensure autonomy of evaluations including outsourcing evaluation to external actors, providing flexibility to evaluation teams in setting the evaluation plans and establishing ethical and professional standards. In most OECD countries at least one of these mechanisms is used, the most common approach is to commission evaluations to external actors (74%).
	External scrutiny and/or quality criteria	In development	In Chile, external peer reviews are conducted for internal evaluations. In 58% of OECD countries there are no peer review mechanisms in place. There are no pre-defined quality criteria to assess the quality of evaluations. 77% of OECD countries do not have pre-define quality criteria.
	Capacity building for skills development	Opportunity for improvement	There are no measures to acquire or retain adequate skills for policy evaluation. 48% of OECD countries provide regular professional updates and trainings.
	Data access for evaluation	Opportunity for improvement	There is no national government strategy to promote the availability and accessibility of data for evaluation and evidence-informed policymaking. 29% of OECD countries have a data strategy that includes promotion of access to data for evaluation and policy analysis.
Measures to ensure impact of policy evaluation	High level guidance	Opportunity for improvement	No high-level guidance framing policy evaluation across the government. 45% of OECD countries have guidance in the form of legal framework.
	Use of evaluation results	At standard	Evaluation results are used in budgetary decision making and to improve programme implementation as in most OECD countries.
	Follow-up mechanisms	At standard	Follow-up mechanisms are present for some evaluations based on commitment from ministries. Across OECD countries, follow-up mechanisms are in place for all evaluations (23% of cases) or for some evaluations (48%).
	Publicity of evaluation results	At standard	Evaluations are public by default as in 61% of OECD countries. Chile also has a central platform with all evaluations published. At the moment, a central platform exists in 42% of OECD countries.
	Evaluation evidence presentation	At standard	Evaluations are published in tailored and accessible formats (e.g. policy briefs, executive summaries, etc.). Only 29% of OECD countries ensures this for all evaluations.
	Evidence synthesis use	Opportunity for improvement	Evaluation results are not aggregated in systematic manner to provide aggregated results. Only 39% of OECD countries uses synthesis methodologies to aggregate findings.