Given its importance, why have you not included equity as its own criteria?
Where equity fits was a key area of discussion as we reflected on how the criteria are used and thought about how they could be improved. Because reducing inequalities and leaving no one behind are policy goals for nearly all governments and other development actors today, we are seeing more interventions aimed at equity, and more interest in evaluating whether or not particular interventions are helping to achieve those goals. Aspects of equity are covered in evaluations as part of relevance, effectiveness and impact – as well as under sustainability, coherence and efficiency.
Some of the changes made to the definitions are also aimed at encouraging better analysis of equity effects. Specifically:
Under relevance, we put the priorities of individuals affected by the intervention up front
Relevance is defined as responding to the equity conditions in context
The definition flags that there may be conflicts between different stakeholders’ priorities and needs (with marginalized groups often having been excluded from official processes to set priorities)
Under effectiveness, the definition now includes differential results across groups
The forthcoming guidance will go into detail on how to apply an equity lens to the criteria
We also think equity needs to be taken into account beyond the criteria: it should inform the entire evaluation process, including decisions about what gets evaluated and how. More work needs to be done to take an equity-informed approach to evaluation.
Are these criteria just for evaluating projects?
No. We use the term “intervention” throughout the definitions because it is a broad, neutral term that can be applied to any evaluand (subject of evaluation). The criteria are commonly used for evaluating policies, strategies, thematic areas, peacebuilding activities, programmes, country programmes/strategies, etc.
Can these criteria be used to evaluate climate change action and finance?
The criteria are useful for evaluating efforts (whether national, sub-national or international) aimed at achieving the Sustainable Development Goals as set out in Agenda 2030 and national contributions to the Paris Agreement. These criteria can be used both to evaluate the extent to which environment and climate change are well-integrated into development interventions (mainstreaming), and to evaluate specific climate mitigation or adaptation interventions.
The revision of the evaluation criteria strengthens the emphasis on environmental effects in several ways, and we believe they will be useful to push for better evaluation of Paris-related commitments, as well as for applying a climate lens to evaluations of development and humanitarian activities. References to environmental effects have been added to the definitions of Relevance, Impact and Sustainability. Furthermore, Coherence is being added as a new criteria to encourage better responses to the climate emergency, including analysis of potential synergies and trade-offs between development and climate priorities.
Can these criteria be used to evaluate transformational change?
Yes and no. Yes, the criteria can be used to understand how different interventions are (or are not) creating transformational changes. The potential transformative effects of an intervention are captured under the new definition of impact. We expect evaluations will look more and more at questions related to transformation, reflecting the growing interest in understanding transformational change and the imperative for transformation described in both Agenda 2030 and the Paris Agreement. The evaluation methods needed to answer these questions are still emerging.
However, the evaluation criteria are intended to be used in evaluating an intervention (some intentional effort to create a change in the world), they are not a particularly useful tool for descriptive analysis of transformative change or systems change.
What about the SDG’s and Agenda 2030 - I don’t see any references in the definitions?
The criteria are intended to be used to evaluate interventions aimed at achieving goals such as those set out in Agenda 2030. They are commonly used to evaluate progress towards achieving SDG targets, by examining the effects of policies, programmes or projects. The criteria are therefore useful for evaluating national or international SDG-related efforts, and they will also remain relevant and useful when policy priorities and goals change.