Share

Evaluation of development programmes

Evaluation Criteria

 

 …/ DCD / _Styles / 00: T4 backOffice(TinyMCE) - 2019
 …/ DCD / _Styles / 02: DCD documentType(DT) styles 2019

The OECD DAC Network on Development Evaluation (EvalNet) has defined six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact and sustainability – and two principles for their use.

These criteria provide a normative framework used to determine the merit or worth of an intervention (policy, strategy, programme, project or activity). They serve as the basis upon which evaluative judgements are made.

Newly adapted definitions and principles of use
are available in multiple languages:
English French Chinese Spanish

German

 

Two principles for using evaluation criteria

It is important that the definitions of the criteria are understood within a broader context, and read in conjunction with other principles and guidance on how to conduct evaluations in ways that will be useful and of high quality

Principle One

The criteria should be applied thoughtfully to support high quality, useful evaluation.

They should be contextualized – understood in the context of the individual evaluation, the intervention being evaluated, and the stakeholders involved. The evaluation questions (what you are trying to find out) and what you intend to do with the answers, should inform how the criteria are specifically interpreted and analysed.

Principle Two

The use of the criteria depends on the purpose of the evaluation. The criteria should not be applied mechanistically.

Instead, they should be covered according to the needs of the relevant stakeholders and the context of the evaluation. More or less time and resources may be devoted to the evaluative analysis for each criterion depending on the evaluation purpose. Data availability, resource constraints, timing, and methodological considerations may also influence how (and whether) a particular criterion is covered.

 

The six evaluation criteria explained

RELEVANCE

IS THE INTERVENTION DOING THE RIGHT THINGS?

The extent to which the intervention objectives and design respond to beneficiaries , global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change.

Note: “Respond to” means that the objectives and design of the intervention are sensitive to the economic, environmental, equity, social, political economy, and capacity conditions in which it takes place. “Partner/institution” includes government (national, regional, local), civil society organisations, private entities and international bodies involved in funding, implementing and/or overseeing the intervention. Relevance assessment involves looking at differences and trade-offs between different priorities or needs. It requires analysing any changes in the context to assess the extent to which the intervention can be (or has been) adapted to remain relevant.

* Beneficiaries is defined as, “the individuals, groups, or organisations, whether targeted or not, that benefit directly or indirectly, from the development intervention." Other terms, such as rights holders or affected people, may also be used.

COHERENCE

HOW WELL DOES THE INTERVENTION FIT?

The compatibility of the intervention with other interventions in a country, sector or institution.

Note: The extent to which other interventions (particularly policies) support or undermine the intervention, and vice versa. Includes internal coherence and external coherence: Internal coherence addresses the synergies and interlinkages between the intervention and other interventions carried out by the same institution/government, as well as the consistency of the intervention with the relevant international norms and standards to which that institution/government adheres. External coherence considers the consistency of the intervention with other actors’ interventions in the same context. This includes complementarity, harmonisation and co-ordination with others, and the extent to which the intervention is adding value while avoiding duplication of effort.

EFFECTIVENESS

IS THE INTERVENTION ACHIEVING ITS OBJECTIVES?

The extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups.

Note: Analysis of effectiveness involves taking account of the relative importance of the objectives or results.

EFFICIENCY

HOW WELL ARE RESOURCES BEING USED?

The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.

Note: “Economic” is the conversion of inputs (funds, expertise, natural resources, time, etc.) into outputs, outcomes and impacts, in the most cost-effective way possible, as compared to feasible alternatives in the context. “Timely” delivery is within the intended timeframe, or a timeframe reasonably adjusted to the demands of the evolving context. This may include assessing operational efficiency (how well the intervention was managed).

IMPACT

WHAT DIFFERENCE DOES THE INTERVENTION MAKE?

The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.

Note: Impact addresses the ultimate significance and potentially transformative effects of the intervention. It seeks to identify social, environmental and economic effects of the intervention that are longer term or broader in scope than those already captured under the effectiveness criterion. Beyond the immediate results, this criterion seeks to capture the indirect, secondary and potential consequences of the intervention. It does so by examining the holistic and enduring changes in systems or norms, and potential effects on people’s well-being, human rights, gender equality, and the environment.

SUSTAINABILITY

WILL THE BENEFITS LAST?

The extent to which the net benefits of the intervention continue, or are likely to continue.

Note: Includes an examination of the financial, economic, social, environmental, and institutional capacities of the systems needed to sustain net benefits over time. Involves analyses of resilience, risks and potential trade-offs. Depending on the timing of the evaluation, this may involve analysing the actual flow of net benefits or estimating the likelihood of net benefits continuing over the medium and long-term.

Applying Evaluation Criteria Thoughtfully

This guidance complements the criteria definitions and their principles for use to support thoughtful and contextualised application of the criteria. The guidance will assist readers as they consider how to interpret and apply the criteria to improve the quality of their evaluations and better support learning and accountability.

Related resources

For further information:

See frequently asked questions | Submit your own question on Twitter | Contact  DACEvaluation.contact@oecd.org

 

Related Documents