Handbook - Developing Partnerships with Non-OECD Economies - Section 2.2

2. EVALUATION AND IMPACT

2.2 Surveys of Experts and

Since 2001, the effectiveness of the programme has been subjected to independent evaluation by the Independent Evaluation Services.  The Independent Evaluation Service (IES) consists of the Canada Revenue Agency and the UK Inland Revenue.  They provide an annual report which makes recommendations on all important issues concerning the organisation and delivery of OECD events.

The feedbacks received from the IES evaluation surveys are critical to the continual improvements of the quality and effectiveness of CFA’s partnership activities.  These feedbacks, which are discussed annually at the Advisory Group on Co-operation with Non-OECD Economies, have led to a number of significant improvements in recent years.  The improvements include the introduction of new facilities in the form of China’s Yangzhou Tax Institute and the development of this online guide on CFA’s partnership activities to facilitate the participant and expert selection process. 

The reports that IES compile every year are based on the evaluation of OECD events by the participants, event leader and OECD experts who take part in OECD events.

2.2.1 Feedback from Participants

It is part of the demand driven philosophy that participants are invited to express their comments and provide inputs into the content of OECD events.  At the end of the event we will take some time to do a more structured evaluation with special forms (see Annex 1).

The event leader is responsible for seeing that this evaluation is carried out.  The evaluation forms should be distributed around lunch time on the second last day of the event and collected at the beginning of the morning on the last day to allow the interpreters sufficient time to translate the written comments if necessary. 

2.2.2 Feedback from Event Leader and Experts

The formal evaluation by the Event Leader and Experts occurs at the end of each event.  Forms for the purpose of this evaluation will be sent by the support staff at the Secretariat to the Event Leader and Experts involved in delivering an OECD event (see Annexes 2 and 3).  The feedback collected from the event leader and experts are sent to the IES so that their comments are collated and incorporated into the annual reports on evaluation.

The team of experts may also wish to conduct an informal evaluation during the second day of an event to obtain feedback on how the event is proceeding and whether any adjustments in the syllabus and presentations need to be made.  It is a matter of judgement for the experts and ultimately the event leader as to when this evaluation is administered and what changes are made as a result.

The regular contacts between the Event Leader and Experts working together to deliver an event provide an ideal opportunity to discuss the event materials well in advance.  The Event Leader will ensure that comments from experts are incorporated and materials are updated and revised where necessary.

2.2.3 The 2004 IES Report

The IES noted in the 2004 report that the approach of organising a mixture of multilateral, regional and bilateral events is a cost effective way of meeting the objectives of the partnership programme.  However, to further improve the impact and cost effectiveness of the program, the IES recommended a number of areas for improvements:

• Although there are some evidences of improvement in participant selection, this aspect continues to be an area that could be further improved.  The Group agreed that more detailed description of these differentiated events should be put on the online handbook to facilitate the participant selection process for multilateral events.
• On event content and materials, the Secretariat is encouraged to liaise with experts to keep event materials updated, to encourage the use of case studies and to improve the quality of translation, subject to budget constraints.
• The quality of facilities where the OECD events are held remain very high.  However, plans by Hungary to make improvements to the facilities of the Budapest Multilateral Tax Centre are welcomed by the IES and the Advisory Group.
• Distribution of some event information (syllabus, background information and some case study materials) prior to the OECD, possibly by putting these on the online handbook site, could assist in improving the effectiveness of the partnership program.