Share

Evaluation

 

These key principles provide guidance and inspiration for International Organisations wishing to develop a greater culture of evaluation. 

1. Institutionalise the evaluation of instruments

Institutionalising the systematic evaluation of (normative) instruments developed by the IO is an important step towards ensuring their continued relevance. The level of formality of such institutionalisation can vary – for example, ‘institutionalising’ an evaluation commitment could mean including evaluation practices in the IOs rules of procedure, or via the creation of a unit whose job it is to carry out evaluations. Either way, it is a demonstration of the commitment of the IO to the continual improvement of its instruments and to ensuring that they remain fit-for-purpose.

 

2. Start small and build evaluation practices over time

IOs find it challenging to evaluate their instruments for a number of different reasons (see section 5). Indeed, it may not even be possible to effectively evaluate every kind of IO instrument. For a type of instrument that has not been evaluated before and for which no evaluation best practice can easily be identified, it makes sense to first assess the ‘evaluability’ of the instrument – looking at its objectives and considering how it is implemented and by whom, will evaluation be possible and/or useful? In most cases, the answer will be ‘yes’, but the scope and breadth of evaluation may differ.

 

3. Develop guidance for those undertaking the evaluation

Developing guidance documents aimed at those responsible for planning or undertaking the evaluation will help to harmonise practices and set expectations for the IO and its stakeholders. A common approach is especially important if evaluations are carried out in a decentralised manner (for example, not led by the IO secretariat, but conducted by members or by external consultants).

 

4. Establish objectives for IO instruments to be evaluated against

When IO instruments have clearly measurable objectives, these serve as helpful criteria for the evaluation. However, when not feasible or leading to an incomplete understanding of the instrument, it becomes important to provide qualitative descriptions of those impacts that are difficult or impossible to quantify, such as equity, fairness, and distributional effects. Depending on the nature of the instrument and the level of evaluation foreseen, objectives might be specific to one instrument, or could apply to a set of instruments or type/class of instruments. Alternatively, the objectives for the instrument may be set (or modified) by the State/organisation implementing the instrument, according to local circumstances.

5. Promote the evaluation of sub-sets or the overall stock of instruments

The IOs should consider evaluating their instruments on more than an individual basis. Evaluating a sub-set of instruments or the whole stock of IO instruments can introduce greater strategic direction into the practices of the IO by providing a detailed overview of the range of instruments applied and lessons on which instruments work better than others.

 

6. Be transparent about evaluation processes and results

The open availability of information about evaluation processes and the transparent dissemination of evaluation results are important to build trust and to demonstrate that your IO has a sound culture of evaluation and of accountability for its instruments.

 

7. Use the results of evaluations

Not only should the results of evaluations be used, but the IO should be able to show how they have been used by the IO, its respective governing bodies, its members or other stakeholders to:

  • Improve IO instruments and/or their implementation, including closing regulatory gaps in the stock of instruments.
  • Identify follow up actions related to other IO practices and items that need to be fed into the next cycle of IO decision-making.
  • Identify lessons-learned during the evaluation process that can improve the evaluation process itself (e.g. improving guidance documents, objective-setting processes or communication of results).
  • Advocate the value of international instruments (see Chapter 2 Strengthening the Implementation of International Instruments).

 

Related Documents