Share

Evaluation of development programmes

DAC Criteria for Evaluating Development Assistance

 

  …/ DCD / Styles / 00 : DCD _backOffice styles (tinyMCE)
  …/ DCD / Styles / 00 : DCD documentType styles 2017
The DAC Network on Development Evaluation is currently exploring how the DAC Evaluation Criteria can be adapted to the new development landscape and the 2030 Agenda. The criteria are widely used in evaluations of development programmes far beyond the membership of the DAC.

Global Consultation on Adapting the Evaluation Criteria

 

In the context of a broader debate about the future of development evaluation, a discussion has begun on re-thinking the five DAC evaluation criteria of relevance, effectiveness, efficiency, impact and sustainability, first articulated alongside evaluation principles in 1991. Throughout 2018, the Network and its partners hosted a global consultation to explore the strengths and weaknesses of the criteria, and how they might best be adapted to today’s development context. Our overall goal – in line with our mission – is to support better evaluation, which will in turn play a role in helping to achieve the Sustainable Development Goals.


The consultation ran from March to October 2018 and included:

 

  • Interviews with key stakeholders;
  • A consultation workshop (March 2018)
  • Discussions at international meetings/seminars in Asia, Africa and Europe;
  • Discussions within meetings of our sister networks UNEG and ECG
  • OECD DAC Network member survey
  • Public survey with stakeholders.

 

We received an impressive number of 691 responses to the public survey! Overall the conversations produced more than 700 pages of input. Thank you to everyone who participated!


Summary of the stakeholder consultation results.


We learned a lot during this consultation. We knew the criteria were widely used, but we were impressed by just how widely they are known and applied in evaluations. The criteria are being used far beyond the membership of the DAC to evaluate programmes and policies outside of development co-operation, including for national and local evaluations across the globe. While there was broad agreement on the strengths of the criteria (their simplicity, clarity and broad applicability were especially appreciated), respondents also saw room for improvement and clarification. Many perceived challenges that were raised have to do with how the criteria are applied, more so than with the criteria themselves.


Building on these learnings, the Network decided to:

  • Draft updated definitions for the original criteria, including “dimensions” to further explain each criterion
  • Consider adding criteria
  • Develop a guidance on how to use the criteria
  • Explore what other steps are needed (beyond adaptation of the criteria) to support better evaluation in today’s context.

This work is ongoing, in close collaboration with our members and external partners.


We will be updating this website regularly. Material from the consultation website (dacevaluationcriteria.org) will soon be available here.

 

In the meantime you can follow the debate on Twitter.

The criteria

When evaluating programmes and projects it is useful to consider the following criteria. The criteria were first laid out in the DAC Principles for Evaluation of Development Assistance and later defined in the Glossary of Key Terms in Evaluation and Results Based Management. The following further explains the criteria and provides some sample questions to illustrate how they may be used in practice:

Relevance

The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.


In evaluating the relevance of a programme or a project, it is useful to consider the following questions:

  • To what extent are the objectives of the programme still valid?
  • Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
  • Are the activities and outputs of the programme consistent with the intended impacts and effects?

Effectiveness

A measure of the extent to which an aid activity attains its objectives.


In evaluating the effectiveness of a programme or a project, it is useful to consider the following questions:

  • To what extent were the objectives achieved / are likely to be achieved?
  • What were the major factors influencing the achievement or non-achievement of the objectives?

Efficiency

Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted.


When evaluating the efficiency of a programme or a project, it is useful to consider the following questions:

  • Were activities cost-efficient?
  • Were objectives achieved on time?
  • Was the programme or project implemented in the most efficient way compared to alternatives?

Impact

The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions.


When evaluating the impact of a programme or a project, it is useful to consider the following questions:

  • What has happened as a result of the programme or project?
  • What real difference has the activity made to the beneficiaries?
  • How many people have been affected?

Sustainability

Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.


When evaluating the sustainability of a programme or a project, it is useful to consider the following questions:

  • To what extent did the benefits of a programme or project continue after donor funding ceased?
  • What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?

Sources: The DAC Principles for the Evaluation of Development Assistance, OECD (1991), Glossary of Terms Used in Evaluation, in 'Methods and Procedures in Aid Evaluation', OECD (1986), and the Glossary of Evaluation and Results Based Management (RBM) Terms, OECD (2000).