This chapter sets out operational actions to support countries in evaluating anti-fraud strategies. It outlines key evaluation questions by adapting the OECD-DAC evaluation criteria to anti-fraud strategies, provides guidance on the timing and methods of evaluations, and defines governance and institutional responsibilities for conducting them. The chapter emphasises the importance of identifying stakeholders, roles, and decision-making processes early in the evaluation process, developing a Theory of Change during strategy design, and assessing both the financial and non-financial impacts of fraud to support evidence-based anti-fraud initiatives.
Evaluating, Updating and Monitoring Anti‑Fraud Strategies
A Methodology
Abstract
While monitoring enables those charged with strategy or project implementation and oversight to assess if they are advancing in the achievement of objectives, strategy evaluation considers whether the desired change has been achieved. An evaluation therefore goes beyond the implementation and achievement of outputs to consider the change to which the strategy has contributed and to what extent the desired results have been achieved and how, considering questions such as relevance, coherence, effectiveness, efficiency, impact or sustainability. In the case of anti-fraud strategies, the purpose of evaluation is to ensure a strategy is well designed to “provide assurance regarding the achievement of operational objectives in the effectiveness and efficiency of the fight against fraud and the protection of Union’s financial interests.” (OECD, 2021[1]; European Commission, 2016[2])
While strategy evaluations can serve as useful tools to inform policymakers on whether an anti-fraud strategy is able to reach its goals, achieve the desired results and is cost-effective and efficient, robust evaluations necessitate methodological rigour and good governance of the process. This entails addressing common challenges in conducting strategy evaluations, such as securing the necessary human resources (capacity and capabilities), ensuring political interest and demand, financial resources, and available high-quality data to produce robust evaluations (OECD, 2020[3]).
This chapter aims to support countries in conducting evaluations of anti-fraud strategies. It includes key questions to consider for strategy evaluation at the anti-fraud strategy development phase, institutional responsibilities for strategy evaluations, and how to effectively communicate the results following evaluation exercises. An overview of the operational actions included in the evaluation workflow and presented in more detail in this chapter is provided in Box 2.1 below.
1. Decide how the evaluation will be managed – including clarifying stakeholders, roles and decision-making processes, and transparency of these processes
2. Determine the purpose and period of the evaluation – is the anti-fraud strategy effective in achieving its objectives? What period is the evaluation covering (mid-term or end-term)?
3. Select the evaluation criteria, the standards to be used and formulate the key questions to consider
4. Plan the evaluation process by setting timeframes and deadlines, reviewing responsibilities, mapping data sources, selecting methods for information gathering and determining the resources for conducting the evaluation
5. Collect and review data from qualitative and quantitative sources to answer the evaluation questions, including annual/bi-annual monitoring reports, administrative data, audit reports and recommendations, outcome and impact indicators, focus groups and surveys
6. Assess performance against the evaluation criteria by synthesising the data, including considerations on the financial and non-financial impact of the anti-fraud strategy, the return on investment and reviewing the intervention logic (Theory of Change) to understand the causes of outcomes and impacts and develop evaluative judgments
7. Draw preliminary conclusions and recommendations, acknowledging any limitations and consulting preliminary findings with relevant anti-fraud authorities
8. Incorporate feedback on the preliminary conclusions and recommendations into the final evaluation report
9. Draft the final evaluation report and communicate results with internal and external stakeholders, including civil society and non-governmental organisations
10. Update the anti-fraud strategy and/or action plan based on the findings from the evaluation exercise
Source: OECD Secretariat; Adapted from (BetterEvaluation, 2025[4])
2.2.1. Determining the criteria for strategy evaluation
There are a number of questions evaluators of an anti-fraud strategy may want to consider to gather information on whether the strategy is achieving the desired results. Ultimately, evaluators want to know whether the anti-fraud strategy is effective in achieving its objectives and reducing fraud, which requires gathering data to identify the factors that are influencing both the successful achievement and non-achievement of anti-fraud objectives.
The OECD Development Assistance Committee (DAC) translates key evaluation considerations into six evaluation criteria (impact, effectiveness, efficiency, sustainability, coherence and relevance) to support consistent, high-quality evaluation of interventions, including strategies, to inspire more effective strategy design in the future. The criteria are intended to prompt evaluators to consider the nature of the strategy, its implementation process and its results in more depth, while acknowledging that the use of the criteria depends on the purpose of the evaluation (OECD, 2021[1]). A brief explanation of each criteria in relation to evaluations of anti-fraud strategies is provided below.
Impact
Impact refers to the extent to which the anti-fraud strategy has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects (OECD, 2021[1]). Evaluators may therefore consider whether the strategy has led to enduring changes in systems and norms related to anti-fraud, and what intended and unintended effects the strategy has generated. This may involve looking at system-wide changes such as a sustained reduction in overall fraud rates, deterrent effects on fraudulent behaviour, shifts in fraud culture and tolerance to fraud, and increased overall trust in anti-fraud institutions. Assessing both financial and non-financial impacts is essential to demonstrate the value of anti-fraud efforts, inform resource allocation, and complement assessments of efficiency, including the return on investment of anti-fraud measures. Given the inherently hidden nature of fraud, evaluating impact can be challenging. Nevertheless, a range of approaches can be used to assess both financial and non-financial impacts of anti-fraud strategies, as discussed later in this chapter.
Effectiveness
Effectiveness considers the extent to which the anti-fraud strategy achieved, or is expected to achieve, its objectives and results, including any differential results across groups (OECD, 2021[1]). This involves assessing whether the anti-fraud strategy is achieving its objectives and meeting targets, including gathering data to identify the major factors influencing the achievement or non-achievement of the objectives. The latter may also shed light on other evaluation criteria, including factors such as a lack of resources to achieve objectives, or having measures that are not appropriately linked (relevant and proportionate) to achieving objectives. To understand whether an anti-fraud strategy has been effective in achieving desired outcomes, evaluators may consider questions such as increased rates in fraud detection, improved or more efficient enforcement outcomes and procedures, and a reduction in fraud losses.
Efficiency
Efficiency considers the extent to which the anti-fraud strategy delivers, or is likely to deliver, results in an economic and timely way (OECD, 2021[1]). Aspects such as a lack of resources (human, financial, time) could be underlying causes as to why an anti-fraud strategy is not delivering the expected results. This could be the case if the necessary financial resources needed to implement the strategy have not been accounted for from the start (financial sustainability), or if objectives and measures are not realistic given the implementation timeline or available human or financial resources.
Sustainability
Sustainability refers to the extent to which the net benefits of the anti-fraud strategy continue or are likely to continue (OECD, 2021[1]). Evaluators should consider whether the benefits resulting from measures adopted under the anti-fraud strategy will last (i.e. are sustainable). This involves looking at financial, economic, social and environmental aspects, as well as any interlinkages between them. For instance, a measure under an anti-fraud objective aiming to increase institutional anti-fraud awareness among public officials that specifies the delivery of a single anti-fraud awareness training to public officials may indeed result in an increased anti-fraud awareness among participating officials. However, the benefits (increased anti-fraud awareness) may not last, as the measure does not account for staff turnover and knowledge retention. A more sustainable option would be to provide recurring anti-fraud awareness trainings (e.g. annually) or to develop a mandatory e-learning training with refresher courses every number of years.
Coherence
Coherence considers the compatibility of the anti-fraud strategy with other interventions in a country, sector or institution (OECD, 2021[1]). An often-overlooked aspect of the strategy development phase is considering whether the strategy is aligned with other governance reforms and policies in relevant key areas, supporting strategy coherence. By considering the synergies, trade-offs, and potential tensions between policy areas, a strategy is better able to ensure that strategic efforts are adding value and complementary, harmonised, and co-ordinated with others, while avoiding duplication of effort (OECD, 2021[1]). While it is important to recognise and distinguish between fraud and other types of integrity violations to apply appropriate preventive and detective controls, fraud does not operate in a silo, and oftentimes fraudulent and corrupt practices occur together. According to the ACFE’s latest Global Fraud Survey, fraudsters had committed both fraud and corruption in 35% of cases (out of 1 921 cases) (ACFE, 2024[5]). This highlights the importance for evaluators to consider synergies between an anti-fraud strategy with any existing integrity or anti-corruption strategies. For instance, in assessing how aligned the anti-fraud strategy is with other governance reforms and policies in relevant areas, the evaluator could consider the NAFS’ synergies with the objectives in a respective anti-corruption strategy to identify duplication of efforts, fragmented communication channels or awareness and data gaps. An example of considering strategy coherence in developing an anti-fraud strategy is provided in Box 2.2.
Portugal’s NAFS for the period 2023-2027, Order No. 7833/2023 of 31 July 2023, considers the objectives of the National Anti-Corruption Strategy (2020-2024). The NAFS sets out that the priorities and fundamental actions to be foreseen under the NAFS will have to be combined and co-ordinated with those defined in the National Anti-Corruption Strategy, acknowledging that both strategic documents consider objectives that are consistent with each other and achievable through the application of related measures.
Source: Adapted from (Presidency of the Council of Ministers and Finance in Portugal, 2023[6])
Relevance
Relevance considers the extent to which the intervention’s objectives and design respond to global, country or institutional needs, policies and priorities, and continue to do so if circumstances change (OECD, 2021[1]). An anti-fraud strategy may not deliver the expected results if the objectives and measures under the strategy are not risk-informed and rooted in a comprehensive problem analysis at the strategy development stage. For instance, an anti-fraud strategy that is not informed by a robust fraud risk assessment may not be adequately addressing or mitigating current and emerging fraud risks. Equally, an anti-fraud strategy that has not been developed in an inclusive and transparent manner (e.g. undergone mandatory inter-institutional and public consultation processes, OECD PII 3.4 on inclusiveness and transparency of intergovernmental and public consultations) may not sufficiently respond to beneficiaries’ needs, policies, and priorities related to addressing fraud.
These considerations underscore the importance of developing a solid ToC or results chain at the strategy development stage. The ToC should outline how planned activities under the anti-fraud strategy will lead to the desired results (outcomes) and long-term changes (impacts), including identifying the underlying assumptions about how each activity will bring about changes towards achieving the strategic goals of the strategy (Georgieva-Andonovska, 2016[7]). If preconditions such as political will or financial or human resources required to implement activities under an anti-fraud strategy have not been considered, it is likely that the anti-fraud strategy is not delivering the desired outcomes.
A summary of the DAC Evaluation criteria, including the key questions to consider when evaluating achievement of outcomes of anti-fraud strategies, are included in Table 2.1 below. Furthermore, Annex A includes a sample questionnaire that evaluators may use for anti-fraud strategy evaluations (mid-term or end-term), with key questions to gather information about the relevance, coherence, effectiveness, efficiency, impact and sustainability of the anti-fraud strategy.
2.2.2. Timing of evaluations and securing the necessary resources early on
While monitoring is a periodical exercise that stretches throughout the strategy cycle, evaluations typically occur at the beginning, mid-term, and end of the strategy implementation cycle, when it is most useful to receive information about the status or progress of a strategy. With this said, the timing of evaluation activities may also depend on when measures under the anti-fraud strategy are expected to have an impact. For instance, ex ante evaluations are useful to conduct before-and-after comparisons and to assess whether a strategy is well-designed and likely to achieve its objectives before it is put into practice; mid-term evaluations focus on performance by enabling identification and adjustment of how a strategy is implemented; and end-term or ex post evaluations are key to inform design processes around future strategies (Johnsøn, 2011[8]). As discussed in Chapter 1, planning evaluations at the onset of the strategy cycle is key, as it enables:
obtaining a reliable baseline to establish trends over time against indicators
scheduling the mid-term evaluation at a time when it will still be possible to make a course correction if the strategy is not meeting its targets and/or broader objectives
ensuring that there will be enough funds at the end of the project to conduct the final evaluation
The European Commission recommends periodic evaluations of NAFS within specified timeframes, including a possible mid-term evaluation, annual evaluation or a final evaluation (at the end of the implementation period of the NAFS). For each evaluation, a report would be produced that describes the methodology and findings concerning progress towards the achievements of the identified operational objectives, as well as next steps (European Commission, 2016[2]). Against this background, it is advisable for the owners of an anti-fraud strategy to list mid-term and end-of-term evaluations as an activity in their action plan, including ensuring that the budget for implementing the strategy covers this exercise.
Source: OECD Secretariat.
2.2.3. Approaches for evaluating the impacts of an anti-fraud strategy
One of the aims of an evaluation is to assess the causal impacts of a strategy. Assessing the impact of an anti-fraud strategy is inherently challenging, given the hidden nature of fraudulent activity. Nevertheless, analysing both the financial and non-financial impacts of anti-fraud strategies is essential for advancing evidence-based approaches to anti-fraud initiatives. In turn, this contributes to strengthening the basis for evaluating the strategy’s impact over time. This sub-section explores approaches to measuring the financial and non-financial impacts of anti-fraud strategies, which can support efforts to demonstrate the value of anti-fraud efforts, prioritise resource allocation, and demonstrate measurable returns on investment of anti-fraud measures.
Estimating the financial impact of fraud
Unlike fraud risk assessments, which identify how or where fraud may occur, fraud measurement seeks to estimate the extent to which fraud is actually taking place and can inform cost-benefit analyses. Estimating the financial impact of fraud is a critical first step in determining whether approaches to tackling fraud are cost-effective, as it establishes a baseline against which the benefits of measures implemented under an anti-fraud strategy can be assessed. This issue is also reflected in various forms in the European Commission’s Guidelines on National Anti-Fraud Strategies, including in the annex on risks related to the functioning of the AFCOS, which highlights challenges such as unreliable reporting systems, information flows, and statistics (European Commission, 2016[2]).
A range of approaches and methodologies exist for estimating the financial impact of fraud. For instance, some public authorities estimate the overall scale of fraud within a given area by applying methods such as statistical sampling, modelling, and benchmarking, helping to illustrate the impact of fraud prevention initiatives. Others set annual targets for reducing or preventing fraud, which supports consistent tracking and measurement over time. Additionally, assessing the non-financial outcomes of anti-fraud initiatives and frameworks enables risk owners to judge how effective current measures are and make adjustments where necessary. Recently, the United States Government Accountability Office published a technical appendix outlining methods for evaluating the effectiveness of fraud risk management in federal programmes, offering practical examples for each element of the framework, including guidance on calculating return on investment and measuring cost savings from fraud prevention. (UK National Audit Office, 2025[9]; OECD, 2020[10]; U.S. Government Accountability Office, 2026[11])
The International Public Sector Fraud Forum (IPSFF) has developed a framework that sets out key principles and processes for conducting Fraud Loss Measurement (FLM) exercises within public sector organisations (Box 2.5). This framework is intended to be used by counter fraud functions across the public sector and aims to support practitioners in producing credible estimates of fraud and error related to specific government programmes, activities, or functions. FLM estimates the overall level of fraud by testing a statistically representative sample of transactions, identifying instances of fraud or error, and extrapolating these findings to the full population to produce an evidence-based estimate of total losses (International Public Sector Fraud Forum, 2025[12]). Information derived from fraud measurement can be used to establish a baseline for the extent of fraud affecting the EU’s financial interests in Member States. This, in turn, can serve as a useful outcome indicator for evaluating the effectiveness of anti-fraud activities under an anti-fraud strategy. It also presents an opportunity to strengthen capacities and methodologies for collecting and consolidating data.
Where undertaking an FLM exercise is not feasible due to capacity, resource or data constraints, anti-fraud authorities could consider undertaking other activities to establish a baseline of fraud, such as reviewing fraud management information from comparable programmes, activities or functions (e.g. historical detection rates, duration and average cost of fraud), collecting and analysing data related to the known amount of fraud and error or conducting a fraud risk assessment exercise to inform the residual risk of fraud and error (i.e. the portion of the inherent risk that remains after considering the controls already in place) (International Public Sector Fraud Forum, 2026[13]).
The IPSFF has developed a framework setting out key principles and processes for conducting Fraud Loss Measurement (FLM) exercises within public sector organisations. The framework is intended to be used by counter fraud functions across all public sector organisations, with a view to supporting counter fraud professionals to conduct FLM exercises that produce credible estimates of the levels of fraud and error related to a specific government programme, activity or function.
The process for conducting FLM exercises is broken down into five stages:
Stage 1 – Area selection
Identify high-risk areas where fraud or error is most likely, using fraud risk assessments and organisational analysis
Prioritise areas with significant residual risk and sufficient available evidence to test for fraud
Consider factors such as total spending, existing fraud intelligence, and effectiveness of current controls
Focus on processes involving expenditure or income flows (e.g. grants, procurement, fees)
Use tools like fraud risk profiling and “heat maps” to target areas where FLM will provide the greatest assurance value
Stage 2 – Identifying residual risks for testing
Conduct or review a Fraud Risk Assessment (FRA) to identify how fraud could occur and where controls may fail
Focus on residual risks, i.e. those that remain despite existing controls
Determine how each risk can be tested, including what data or evidence is needed
Engage stakeholders across the organisation and adopt a “fraudster mindset” to identify potential vulnerabilities
Assess available data (internal and external) for quality, completeness, and relevance, and select a manageable number of risks for testing
Stage 3 – Planning, sampling and gathering initial evidence
Develop a clear testing plan, defining scope, methodology, and decision-making processes
Establish case classifications (e.g. fraud, error, indicators of fraud, unresolved)
Select a statistically valid sample that is representative of the population, using methods such as random, systematic, or stratified sampling
Determine appropriate sample size based on confidence levels, margin of error, and resource constraints
Identify and collect robust evidence, including external data sources where possible, ensuring it is reliable and sufficient to support conclusions
Stage 4 – Conducting the testing
Systematically review each sampled case and assign it a classification based on available evidence
Apply a phased testing approach, starting with initial checks and progressing to deeper investigation where needed
Use additional evidence to resolve ambiguous cases, while recognising that some cases may remain unresolved
Ensure consistent, well-documented decision making and maintain a clear audit trail
Aggregate results into key categories (irregularity, fraud, error, indicators, unresolved) for analysis
Stage 5 – Reporting and next steps
Extrapolate results from the sample to estimate fraud and error levels across the full population, using confidence intervals
Report findings transparently, including methodology, sample size, confidence levels, and financial impact
Provide insights into which fraud risks are materialising, not just overall rates
Use results to inform control improvements, investigations, and financial recovery, where appropriate
Feed findings into a continuous fraud risk management cycle, supporting both tactical actions (e.g. investigations) and strategic improvements (e.g. strengthening controls and prevention measures)
Source: Adapted from (International Public Sector Fraud Forum, 2025[12])
Approaches to establishing a counterfactual
While fraud loss measurement can create a baseline against which measures implemented under an anti-fraud strategy can be assessed, an evaluation should also seek to understand whether there is causality between a specific intervention under the anti-fraud strategy and the financial or non-financial impact (i.e. how much of this impact occurred because of the anti-fraud strategy intervention and not because of other factors?). One method for assessing the causal impact of an anti-fraud strategy is through establishing a counterfactual, which allows evaluators to compare what occurs when a specific intervention of the anti-fraud strategy is implemented to what occurs in its absence. For instance, to assess the impact of newly introduced identity fraud training and controls, managers could compare both the estimated financial exposure and the likelihood of identity fraud occurring before and after implementing the controls. These projected reductions in risk can then be weighed against the cost of delivering the programme to evaluate its overall value. (U.S. Government Accountability Office, 2026[11])
A counterfactual can be established through a variety of well-established methods, where some are more frequently used than others. Broadly, these methods can be grouped into experimental (establishment of a control group or sample), quasi-experimental and non-experimental methods (BetterEvaluation, 2025[14]). Each approach has its own strengths and drawbacks, and there is not a “one size fits all” for impact evaluations. A common method to establish a counterfactual is through randomisation, whereby exposure to a component of an anti-fraud strategy is randomly assigned across observational units or groups. This approach enables the identification of causal effects, as random assignment supports the assumption that the only systematic difference between groups is their exposure to the intervention. In practice, however, naturally occurring counterfactuals are often difficult to identify. As such, experimental, quasi-experimental and non-experimental methods can be used to construct them for evaluation purposes. For example, anti-fraud awareness raising initiatives can be assessed using experimental designs that compare randomly selected groups of citizens, with one group exposed to targeted messaging (the “treatment group”) and another not exposed (the “control group”) (Peiffer and Cheeseman, 2023[15]).
Where randomisation is not feasible, several quasi-experimental statistical methods can be used to approximate randomisation to create a credible counterfactual. Mixing different methods is also possible, where for example randomised control trials and non-experimental methods are used to complement each other. With this said, application of these approaches requires a solid understanding of their underlying assumptions and methodological limitations. Before-and-after comparisons of indicators collected during the baseline and subsequent assessments can also offer compelling estimates of the financial and non-financial impacts of an anti-fraud strategy (e.g. reduction in fraud losses, increased trust in public institutions, increased anti-fraud awareness). (OECD, 2017[16])
Where baseline data are unavailable, comparisons can be made between non-random observational units that are similar across key characteristics but experience different levels of exposure to activities under an anti-fraud strategy (OECD, 2017[16]). Because assignment to these comparison groups is not random, such analyses are particularly susceptible to bias, and any conclusions regarding causal impacts should be interpreted with caution. Table 2.2 and Box 2.4 below summarise the principal impact evaluation methods and their respective limitations.
Source: Adapted from (OECD, 2020[17]; BetterEvaluation, 2025[14]; Partnership for Economic Inclusion, 2022[18]).
Randomised control trials (RCTs) have become the benchmark for evaluating policy interventions, whether in the context of vaccines, educational programmes, or donor-funded development initiatives. Their appeal lies in the ability of randomisation to minimise selection and confirmation biases, while the inclusion of a counterfactual allows evaluators to isolate the effect of the intervention, control for maturation effects, and establish causal relationships with a high degree of confidence.
However, RCTs also have limitations that policymakers should consider when determining their suitability:
Cost and data requirements: RCTs can be resource-intensive, particularly when relevant data are limited or difficult to access. In many cases, some evaluation questions can be addressed using alternative methods. Leveraging accessible, high-quality administrative or public data can reduce the costs of RCTs and other evaluation approaches while still providing robust insights.
Feasibility of counterfactuals: Constructing a valid counterfactual is not always possible in real-world settings. Piloting a programme before scaling up can help, but expansion may introduce additional complexities.
Ethical considerations: In certain policy areas, it may be ethically or politically problematic to treat groups differently, especially when control groups consist of vulnerable populations who could be denied potentially beneficial interventions. In the context of anti-fraud strategies, these concerns are typically less pronounced, as interventions do not withhold essential support from individuals.
Source: Elaborated by the OECD Secretariat, based on the discussions held at the “International Workshop on Rigorous Impact Evaluation Approaches including Randomised Controlled Trials”, which was co-hosted by the OECD and the Australian Centre for Evaluation on 5 February 2025 at the OECD headquarters in Paris.
Demonstrating the return on investment
Calculating the return on investment (ROI) can help evaluators assess how much value has been generated under an anti-fraud strategy, or a specific measure, for every euro spent. As such, it can serve as a useful performance indicator to inform resource prioritisation, support the adaptation of measures, and justify further investment. While cost-benefit analysis provides a comprehensive framework for comparing the costs and benefits of an anti-fraud strategy, ROI offers a simplified metric that expresses these results as a ratio, thereby facilitating communication with decision makers. However, ROI typically captures only monetised impacts and should therefore be interpreted alongside broader evidence on non-financial outcomes.
Estimating ROI requires assessing several key elements, including the value at risk, the likelihood of that risk materialising, and the resulting annual level of risk. It also involves evaluating how a given programme or investment affects both the probability of the risk and its annual impact, alongside the associated implementation costs. Based on these inputs, managers can compare total costs and expected benefits over a defined period in order to calculate ROI (U.S. Government Accountability Office, 2026[11]; International Public Sector Fraud Forum, 2023[19]). An example of how the ROI can be calculated is provided in Box 2.5 below.
Example from the IPSFF’s Fraud Control Testing Framework
Here is an example of how to calculate the future loss prevented through ongoing identity compromise over a 5-year time horizon. To mitigate the threats to client identity information through phishing and social engineering, the department proposes to put service delivery staff through training twice per year and implement regular fraud control testing at a cost of EUR 46 000 per year.
Assessing the logic (Theory of Change) behind the anti-fraud strategy
Considering the underlying ToC when conducting evaluations of anti-fraud strategies is key to understand gaps in the intervention’s design that may have undermined its relevance, such as whether the actions under the anti-fraud strategy are appropriate for the specified institution, realistic, or correspond to identified needs and challenges (OECD, 2021[1]). Evaluators should therefore test whether the underlying assumptions about why objectives and measures under an anti-fraud strategy will lead to the desired results are valid (e.g. “conducting communication campaigns on anti-fraud measures targeting beneficiaries of EU funds will contribute to a culture of integrity, thereby increasing trust in management of funds" or "integrating IT tools and automation will contribute to achieving an enhanced proactive fraud detection"). An anti-fraud strategy that is not rooted in an analysis of the preconditions required to trigger change along the causal pathway (e.g. necessary resources or political will) may not lead to the desired results, necessitating modified measures and/or objectives (Johnsøn, 2012[20]). Where an anti-fraud strategy lacks a ToC, evaluators should work together with relevant anti-fraud authorities to reconstruct or articulate the ToC before the evaluation starts.
An ex post evaluation conducted by the Greek National Transparency Authority on Montenegro’s NAFS for the period 2019-2022 included evaluation questions about the strategy’s theory of change and intervention logic. The evaluation concluded that the design and implementation of the strategy would benefit from the adoption of a clearer and more concrete theory of change, including the necessary resources and inputs, desired outcomes, key stakeholders’ analysis, activities that will lead to the outcomes, causal pathways between outcomes and activities, indicators and the underlying rationale. The evaluation stressed that formulating a clear ToC for the next strategy period would enable a clear framework to assess milestones and track implementation during monitoring and implementation of the NAFS, including planning for adequate resources.
Evaluations should be well-governed as well as technically and methodologically sound. Rigorous evaluation design, robust data collection, and appropriate analytical methods, supported by sufficient resources, are essential for drawing reliable lessons. At the same time, evaluations take place within a political context in which various stakeholders may seek to influence results.
To better understand this, it is useful to distinguish between three types of use:
Symbolic or persuasive use: Results are taken up to justify or legitimise a pre-existing position, without altering it.
Conceptual use: Results contribute to improved understanding or to a change in the conception of the subject under evaluation.
Instrumental use: Recommendations inform decision making and lead to tangible changes in the policy or programme being evaluated, for example, the reallocation of funds following poor performance (OECD, 2020[17]).
Efforts should be made to maximise the likelihood of instrumental use and to ensure that the evaluation process is as independent as possible, while recognising that it is never fully neutral or all-encompassing. This underscores the importance of incorporating multiple perspectives and exercising caution when generalising conclusions (OECD, 2020[17]).
2.3.1. External and/or independent evaluation
Independence is important for objectivity and to enhance credibility (OECD, 2017[16]). There are several structural models that could be put into place to ensure independence. For example, governments may outsource evaluations to third parties such as non-governmental organisations (NGOs). Depending on the context and the policy area, these can be small, local, civil society NGOs or specialised units from international organisations that may review strategies for their sector of expertise. In other cases, an evaluation unit sits within an agency or ministry. There are also instances where evaluation is the responsibility of a separate agency. The advantages and drawbacks of each option should be considered on a case-by-case basis, as systems which solely rely on independent evaluations (i.e. external to the unit) make it challenging to utilise internal expertise, which may be challenging in understanding context. There is also the possibility that the use of an external entity, such as an NGO, may incur additional costs.
2.3.2. The institutionalisation of evaluations
The governance of policy evaluations is shaped by two key elements: the underlying legal framework and the distribution of evaluation functions and responsibilities across government institutions. (OECD, 2020[17]) Regarding the legal framework, there are different arrangements that mandate the use of independent evaluations to inform future policies. For example, the 2018 OECD Survey on Policy Evaluation found that 29 of 42 countries had established a legal basis for policy evaluation (OECD, 2020[3]). This can take several forms, including constitutional requirements to assess the effectiveness of some or all public policies (e.g. Colombia, Costa Rica, France, Switzerland); primary legislation specifically governing policy evaluations, as in Japan or South Korea; or secondary legislation or executive acts providing more detailed policy frameworks such as in Argentina, Brazil, Kazakhstan, Norway, Romania and the Slovak Republic. In addition to making policy evaluations compulsory, legal frameworks often assign institutional responsibilities for evaluation and establish standards to promote the effective use and quality of evaluations. These standards typically cover both methodological and governance aspects, including ethical conduct, stakeholder engagement, reporting, and the use of findings in policymaking. According to the 2018 OECD Survey on Policy Evaluation, 31 countries had such guidelines in place to support the implementation of policy evaluation across government (OECD, 2020[3]).
Countries also employ a range of mechanisms to ensure quality in government evaluations, addressing both technical standards and good governance. These include provisions set out in legal or policy frameworks; cross-government guidelines for policy evaluation; competence requirements for evaluators; internal or external peer reviews; and systematic meta-evaluations. Where available, national instruments to standardise good practices serve as valuable resources. Examples include the Magenta, Green, and Aqua books guiding evaluators in the United Kingdom; the instructions established in South Korea’s Framework Act on Government Performance Evaluation; and data collection standards specified in official documents in France and Norway (OECD, 2020[17]). For good practices derived from cross-country comparisons, guidance developed by international organisations, such as the European Commission, INTOSAI, and the OECD, including the recently published Implementation Toolkit for the Recommendation on Public Policy Evaluation, can also provide useful insights (OECD, 2025[22]).
Practices regarding institutional responsibilities for policy evaluations vary across countries. Nonetheless, the vast majority of surveyed countries reported at least one institution with responsibilities for evaluations across government. The most commonly assigned institutions, in order of frequency, are:
the Centre of Government (Presidency, Prime Minister’s Office, or equivalent)
the Ministry of Finance or equivalent
the Ministry of Public Sector Reform or equivalent, or an autonomous agency
the Ministry of Planning and Development or equivalent
Institutional responsibilities for evaluating anti-fraud strategies
Building on the general principles of evaluation governance and institutional responsibilities, it is important to clarify which actors should be responsible for evaluating anti-fraud strategies. The function co-ordinating the implementation and monitoring of the anti-fraud strategy, such as the AFCOS in EU Member States, would be well suited to co-ordinate strategy evaluations, given its comprehensive overview of anti-fraud activities, access to relevant data, and engagement with key stakeholders across government institutions.
Furthermore, countries may consider empowering a central body with the overarching responsibility to co-ordinate and supervise the evaluation process. A central body could ensure that the relevant information on the overall status of the implementation of the strategy is bundled and analysed jointly to draw a complete and coherent picture that facilitates decision making, communication and provides incentives for improvement through benchmarking. The results of the evaluation could be discussed within the AFCOS to take decisions on whether or how to revise the anti-fraud strategy, with subsequent approval from the supreme executive organs, such as the Council of Ministers. With this said and as previously mentioned, strategy evaluations, or parts of the process, can also be outsourced. The choice of conducting an evaluation in-house or through an outsourced provider depends on the purpose of the evaluation and the resources available, but in short, in-house evaluations can promote a higher organisational self-reflection and lessons learned, whereas an outsourced evaluation can provide for a more objective evaluation by external and independent evaluators (OECD, 2020[23]).
Portugal
In Portugal’s national anti-fraud strategy for the period 2023-2027, according to decree-law No.5/2023 of 25 January, provides that it is incumbent upon the IGF Audit Authority to co-ordinate the processing of information relating to reports of irregularities and to exercise the other powers arising from its designation as the Portuguese AFCOS, including leading the preparation, co-ordination and implementation of the national anti-fraud strategy, within the scope of European funds.
Romania
Romania’s national anti-fraud strategy for 2023-2027 specifies the Department for the fight against fraud (DLAF) as the anti-fraud co-ordination service for the strategy. In this function, DLAF has the obligation to develop, co-ordinate and implement the strategy.
Latvia
For Latvia’s Corruption Prevention and Combating Action Plan for 2023-2025, the Corruption Prevention and Combating Bureau (KNAB) is responsible for the overall implementation of the Anti-Corruption Action Plan, as well as co-ordinating and monitoring the implementation of the measures set out in the Action Plan. Implementing institutions submit information to KNAB on the progress and results of the implementation of the tasks, whereas KNAB submits an evaluation of the implementation of the Action Plan each year.
Bulgaria
Under Bulgaria’s NAFS for 2021-2027, the AFCOS Directorate performs control, information, and co-ordination activities related to the protection of the EU financial interests and acts as the anti-fraud co-ordination service for the NAFS. While the AFCOS Directorate co-ordinates the implementation of the NAFS, it outsourced the mid-term evaluation to the OECD through the Technical Support Instrument of the European Commission. The OECD team worked in close collaboration with Bulgarian anti-fraud authorities to collect quantitative and qualitative data to assess the NAFS’ performance against the DAC evaluation criteria. The mid-term evaluation resulted in a report related to the findings of the mid-term evaluation and a report related to recommendations for the development of an updated NAFS.
References
[5] ACFE (2024), Occupational Fraud 2024: A Report to the Nations, Association of Certified Fraud Examiners, https://www.acfe.com/-/media/files/acfe/pdfs/rttn/2024/2024-report-to-the-nations.pdf (accessed on 13 October 2025).
[14] BetterEvaluation (2025), Compare results to the counterfactual, https://www.betterevaluation.org/frameworks-guides/rainbow-framework/understand-causes/compare-results-counterfactual (accessed on 20 April 2026).
[4] BetterEvaluation (2025), Rainbow Framework, https://www.betterevaluation.org/frameworks-guides/rainbow-framework (accessed on 16 April 2026).
[25] Corruption Prevention and Combating Bureau of Latvia (2023), The Corruption Prevention and Combating Action Plan 2023-2025, https://www.knab.gov.lv/en/media/3900/download?attachment (accessed on 3 July 2025).
[26] Council of Ministers (2020), National Strategy for Preventing and Combating Irregularities and Fraud Affecting the Financial Interests of the European Union over the period 2021-2027.
[2] European Commission, E. (2016), Guidelines on National Anti-Fraud Strategies, European Commission, https://sfc.ec.europa.eu/sites/default/files/EN-ORI-General%20Guidelines%20on%20National%20Anti-Fraud%20Strategies%20ARES(2016)6943965.pdf (accessed on 15 November 2024).
[7] Georgieva-Andonovska, E. (2016), Monitoring and Evaluation of Anticorruption Action Plans, World Bank, https://documents1.worldbank.org/curated/en/129401593200074880/pdf/Monitoring-and-Evaluation-M-E-of-Anti-Corruption-Action-Plans.pdf (accessed on 22 March 2026).
[21] Government of Montenegro (2024), Evaluation Report - NAFS 2019-2022, https://www.gov.me/en/documents/ea9e1c1f-b917-431f-9a15-3b0fe11e1836.
[24] Government of Romania (2023), “National Strategy of December 14, 2023 anti-fraud measures for the protection of the European Union’s financial interests in Romania, 2023-2027”, Official Gazette no. 1.114 bis of December 13, 2023, https://legislatie.just.ro/Public/DetaliiDocumentAfis/277523 (accessed on 2 July 2025).
[13] International Public Sector Fraud Forum (2026), Forthcoming Fraud Prevention Savings Framework.
[12] International Public Sector Fraud Forum (2025), Fraud Loss Measurement Framework, https://www.gov.uk/government/publications/international-public-sector-fraud-forum-guidance/fraud-loss-measurement-framework-html (accessed on 16 October 2025).
[19] International Public Sector Fraud Forum (2023), Fraud Control Testing Framework (FCTF-01), https://www.counterfraud.gov.au/sites/default/files/2023-11/IPSFF-Fraud-Control-Testing-Framework-FCTF-01.PDF (accessed on 21 March 2026).
[20] Johnsøn, J. (2012), “Theories of change in anti-corruption work: A tool for programme design and evaluation”, U4 Anti-Corruption Resource Centre, Chr. Michelsen Institute.
[8] Johnsøn, J. (2011), “How to Monitor and Evaluate Anti-Corruption Agencies: Guidelines for Agencies, Donors, and Evaluators”, U4 Anti-corruption Resource Centre 8, https://www.u4.no/publications/how-to-monitor-and-evaluate-anti-corruption-agencies-guidelines-for-agencies-donors-and-evaluators-2.pdf (accessed on 22 March 2026).
[22] OECD (2025), Implementation Toolkit for the OECD Recommendation on Public Policy Evaluation, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/77faa4fe-en.
[1] OECD (2021), Applying Evaluation Criteria Thoughtfully, OECD Publishing, Paris, https://doi.org/10.1787/543e84ed-en.
[10] OECD (2020), Countering Fraud in Social Benefit Programmes: Taking Stock of Current Measures and Future Directions, OECD Publishing, Paris, https://doi.org/10.1787/71df2657-en.
[3] OECD (2020), How Can Governments Leverage Policy Evaluation to Improve Evidence Informed Policy Making?, OECD, Paris, https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwi5ttvc4KGVAxUnV6QEHYaBKTMQFnoECA0QAQ&url=https%3A%2F%2Flegalinstruments.oecd.org%2Fapi%2Fdownload%2F%3Furi%3D%2Fprivate%2Ftemp%2F7ecfe9af-5b05-4992-9a10-ebdc45a944a8.pdf%26name%3Dpo (accessed on 12 March 2026).
[17] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/89b1577d-en.
[23] OECD (2020), OECD Public Integrity Handbook, OECD Publishing, Paris, https://doi.org/10.1787/ac8ed8e8-en.
[16] OECD (2017), Monitoring and Evaluating Integrity Policies, Working Party of Senior Public Integrity Officials GOV/PGC/INT(2017) Vol. 4, Paris.
[18] Partnership for Economic Inclusion (2022), PEI Impact Evaluation Workshop – Moving Economic Inclusion to Scale, The World Bank, https://thedocs.worldbank.org/en/doc/51cd983ef9c3e14ad79e75729c9cb8e6-0050022022/original/5-Ag-Ignite-Session-Premand.pdf.
[15] Peiffer, C. and N. Cheeseman (2023), “Message misunderstood: Why raising awareness of corruption can backfire”, U4 Brief, U4 Anti-Corruption Resource Centre, Chr. Michelsen Institute.
[6] Presidency of the Council of Ministers and Finance in Portugal (2023), National Anti-Fraud Strategy under European Funds 2023-2027, https://portugal2030.pt/wp-content/uploads/sites/3/2023/07/despacho7833_2023.pdf (accessed on 2 July 2025).
[11] U.S. Government Accountability Office (2026), Combating Fraud: Approaches to Evaluate Effectiveness and Demonstrate Integrity, https://www.gao.gov/assets/gao-26-107609.pdf (accessed on 21 January 2026).
[9] UK National Audit Office (2025), Estimating and reporting fraud and error in annual reports and accounts, https://www.nao.org.uk/wp-content/uploads/2025/02/estimating-and-reporting-fraud-and-error.pdf (accessed on 17 October 2025).