Evaluating, Updating and Monitoring Anti‑Fraud Strategies

Annex C. Checklist for updating anti-fraud strategies and action plan

What is the purpose of the Checklist?

• It supports the NAFS co-ordinator and implementing agencies to determine whether a strategy/action plan needs a revision/update

• It provides an objective & evidence-based response to the question “Is the current strategy or action plan fit for its purposes or is it outdated, incomplete and/or methodologically weak?”

Who should use it?

• NAFS co-ordinator

Key features/components:

• Checklist of diagnostic questions

• Data sources

Tool Complementarity and Interlinkages:

• Informed by the outputs obtained following the application of the Survey for NAFS evaluations and the implementing agencies’ self-assessment

Guidance on application and limitations:

• The checklist is designed to serve as a generic, simple scoring grid targeting implementation status, quality of evidence, level of completion, or degree of results achieved. This tool could be applicable to national stakeholders in deciding on whether the NAFS and Action Plan require an update

• According to the European Commission’s Guidelines on National Anti-Fraud Strategies, updating or readjustment of the NAFS will include, where relevant:

  • Reconsidering the objective

  • Better and more accurate redirection, planning and management of resources

  • Updating the measures and/or the deadlines

  • Reviewing the services responsible (implementing agencies)

• Against this benchmark, indicators used for assessing the impact of the NAFS implementation shall also be used to inform the update and/or revision of the NAFS in due time

• The Checklist is designed based on the assumption that the NAFS is organised around a three-level architecture comprising: Level 1 – Strategic Goals/Objectives; Level 2 – Operational or Specific objectives; and Level 3 – Concrete actions or measures. Accordingly, the terminology used throughout the survey reflects this assumed structure

• The Checklist is also designed based on the assumption that M&E reporting procedures (such as a self-monitoring form or evaluation survey) contain a separate tabular form to capture relevant and additional information that is needed to inform potential updates of NAFS or action plan

• The Checklist’s questions are presented as generic model questions and do not capture the specific institutional, legal, or policy particularities of individual countries. They should therefore be reviewed and tailored to the relevant national context prior to being used for updating a NAFS

A: Checklist for updating the anti-fraud strategy

Updating NAFS may entail:

• Updating and/or revising the objectives

• Adjusting the management resources

• Adjusting the impact and/or outcome indicators

By answering the following questions, the NAFS co-ordinating authority can assess whether an update of the NAFS is needed and, if so, determine the nature and extent of the revision required:

 

Coverage	No	Question	Data sources
These questions seek to determine whether the NAFS is still relevant	1.	Have new fraud risks emerged that are not reflected in the NAFS?	Latest fraud risk assessment update Implementing agencies - self-reporting
	2.	Have some risks decreased or become irrelevant since the NAFS was adopted?	Latest fraud risk assessment update Implementing agencies - self-reporting
	3.	Has the transition to a new EU programming period/preparation for the next MFF introduced new requirements that are not reflected in the current strategy? / Is the current programming period coming to an end?	Programming documents
	4	Have global governance indicators (TI - CPI, Eurobarometer, etc.) shown significant deterioration since adoption?	Governance indicators NAFS Evaluation results National Anticorruption Strategy Evaluation results
These questions seek to account for the main structural and/or organisational changes	5.	Have there been significant legislative changes affecting anti-fraud, irregularities, conflicts of interest, controls, and sanctions?	Official Gazette EU new regulations, directives
	6.	Have there been major institutional changes that affect mandates, co-ordination or implementation responsibilities?	Implementing agencies - self-reporting
	7.	Have new institutional actors become relevant for NAFS implementation, but are not reflected in the current form of the Strategy?	NAFS co-ordination body meeting minutes Mid-term evaluation
	8.	Have any institutions lost legal mandate, functions or capacities assumed in the NAFS?	Implementing agencies - self-reporting NAFS co-ordination body meeting minutes Mid-term evaluation
	9.	Have EU-level anti-fraud instruments or bodies changed in ways that require strategic alignment (e.g. adoption of the new Anti-Corruption Directive)	EU regulatory monitoring
These questions seek to determine whether the intervention logic is strong	10.	Are the Strategic objectives (L1) supported by Operational objectives that reflect real change (L2)?	Mid-term evaluation External assessments (OECD PII, OLAF evaluations)
	11.	Are there any major gaps in the causal chain between outputs, outcomes and impacts?	Mid-term evaluation
These questions seek to determine if NAFS implementation is conducted through adequate quantified indicators	12.	Are there impact-level indicators in place for the Strategic objectives (L1)?	Mid-term evaluation External assessments
	13.	Are there outcome-level indicators in place for the Operational objectives (L2)?	Mid-term evaluation External assessments
	14.	Are the impact and outcome indicators still measurable with available data?	Implementing agencies - self-reporting
	15.	Do the impact and outcome level indicators capture real change or only fulfilment of activities?	Mid-term evaluation External assessments
These questions seek to determine the performance and implementation of NAFS (effectiveness and efficiency)	16.	Has implementation shown that some objectives are unrealistic?	Mid-term evaluation Monitoring reports
	17.	Has the evaluation shown that some measures have a limited contribution to expected outcomes?	Mid-term evaluation

B: Checklist for updating the action plan

Updating the action plan may entail:

• Updating and/or revising the measures

• Introducing new measures

• Adjusting the planning and management resources

• Adjusting the output-level indicators

• Updating deadlines

• Introducing new measures

By answering the following questions, the NAFS co-ordinating authority can assess whether an update of the Action Plan is needed and, if so, determine the nature and extent of the revision required:

 

Coverage	No	Question	Data sources
These questions seek to assess the quality of the design of the Action Plan	1.	Are the measures clearly linked to operational objectives?	Implementing agencies - self-reporting
	2.	Are the measures sufficiently specific, realistic and implementable?	Implementing agencies - self-reporting
	3.	Are the responsibilities, deadlines and expected outputs clearly assigned?	Implementing agencies – self-reporting
These questions seek to determine the implementation progress	4.	Are there any significant delays in implementation?	NAFS monitoring Implementing agencies – self-reporting
	5.	Are some activities repeatedly postponed or left incomplete?	NAFS monitoring Implementing agencies - self-reporting
	6.	Are some institutions systematically failing to implement their assigned measures?	NAFS monitoring Implementing agencies – self-reporting
	7.	Have some activities become obsolete, unnecessary or impossible to implement?	NAFS monitoring Implementing agencies – self-reporting
These questions seek to determine the feasibility of the Action Plan	8.	Are the deadlines in the action plan still realistic?	NAFS monitoring Implementing agencies – self-reporting
	9.	Are the targets in the action plan still realistic?	NAFS monitoring Implementing agencies – self-reporting
	10.	Are there sufficient human, financial and technological resources available to implement the assigned measures?	NAFS monitoring Implementing agencies – self-reporting
	11.	Have external technical assistance (e.g. TSI projects) created new implementation opportunities or arrangements not reflected in the initial Action Plan?	Projects’ co-operation agreements and outputs NAFS monitoring
These questions seek to determine if the Action Plan implementation is conducted through adequate quantified indicators	12.	Are there output-level indicators in place and for each measure (L3)?	NAFS monitoring
	13.	Are the output indicators still measurable?	NAFS monitoring Implementing agencies – self-reporting
These questions seek to determine whether the Action Plan has internal consistency	14.	Are there activities that duplicate each other?	NAFS monitoring Implementing agencies – self-reporting
	15.	Are important implementation steps missing from the plan?	NAFS monitoring Implementing agencies – self-reporting
	16.	Are there measures with weak causal links to operational objectives?	Mid-term evaluation
Other needs for adjustment	17.	Should any measures be added in response to new risks or institutional needs?	Implementing agencies – self-reporting
	18.	Should any measures be merged or removed?	Implementing agencies – self-reporting
	19.	Should responsibilities be reassigned?	Implementing agencies – self-reporting
	20.	Should deadlines or targets be updated?	Implementing agencies – self-reporting