CHAIR’S SUMMARY

Ministers from OECD countries met for an informal discussion on evaluating the outcomes of higher education. In an opening presentation, Morio Ikeda, of the Shiseido Corporation, emphasised the varied and demanding expectations that society has of higher education institutions and systems in the twenty-first century, and the need for OECD countries to respond. OECD Deputy Secretary-General Aart de Geus referred to the increasingly significant role of higher education as a driver of economic growth and the pressing need for better ways to value and develop higher education and to respond to the needs of the knowledge society. I emphasised the opportunity that lay before us to pursue reform so as to focus higher education policy more on quality and not just quantity.
 In discussion we:

• Noted that higher education is expected to produce a range of outcomes -  basic research, technology transfer, the conservation and communication of culture, education for young – and not so young - students, and engagement with wider society – efficiently, fairly and to a high standard; and that Governments and other stakeholders have therefore been turning increasingly to evaluation as a way to meet this challenge;
  

• Exchanged experiences on evaluating the quality of education and research through self-evaluation, peer review and third party evaluation, as well as ideas on how to maximise the benefit of evaluation while acknowledging the challenges of assessing concepts such as innovation and learning

• Agreed that associating the outcomes from evaluation with incentive structures for institutions, including the provision of additional funds, can be a powerful lever for quality assurance and quality improvement and foster competition among institutions, while noting that it may lead to increasing disparity within countries and that there are significant challenges for policy to encourage improvement among institutions at the lower end of the performance spectrum.
  

• Underlined the importance of improving the information base for the evaluation of higher education outcomes and the need to take account of variations in student aspirations, local labour-market contexts, the mission of institutions and the composition of their student intake.
  

• Welcomed the Berlin Principles as a possible framework for the conduct and development of rankings of higher education institutions, and underlined that rankings and international ‘league tables’ are only as valid as the information on which they are based and can lead to distortions in institutional behaviour.

• Agreed that it was less problematic to evaluate research outputs, as the immediate outcomes of institutional performance, than educational outcomes that, ideally, would incorporate aspects of labour-market and social outcomes which become apparent only in the subsequent life of graduates, but considered that the bias in the information base of existing rankings towards research outcomes could detract from efforts to improve educational performance.

• Agreed that evaluation could only be effective if it was linked to consequences for institutions and individuals and that, because the stakes are high for potential students and their employers, governments and other stakeholders should pursue and promote their efforts to provide reliable timely information on outcomes and to make this public, noting that such efforts could improve the quality of the evaluation and ranking of higher education institutions by evaluation agencies and external observers in the media or elsewhere.

• Debated the extent to which an evaluation of educational outcomes might lead to uniformity in higher educational provision and run counter to diversity in objectives, missions, and institutional structures and agreed that any evaluation of higher education performance should provide for multiple dimensions of outcomes.
  

• Underlined the importance of establishing valid and reliable measures of learning outcomes and welcomed the initiative led by the OECD to assess the feasibility of an international study on assessment of learning outcomes, with the aim of contributing to increased accountability and improvement of assessment methods of learning outcomes by governments, institutions and quality assurance agencies; but noted the need for further debate on the potential benefits and risks, and recommended that, while the feasibility study could not address all aspects of learning outcomes, it should aim to take into account the historical, linguistic, and cultural contexts, and the differences that exist between countries in curricula, duration of study and enrolment rates.

• Underlined the need to develop and implement the work in open and transparent ways, to involve higher education institutions and relevant agencies in the process, and to document the conceptual underpinning of the proposed feasibility study, the criteria for success and the process to assess the validity of the measures.
  

• Sought greater detail on the scope of the feasibility study, in terms of the number of institutions and countries to be involved, the subject areas that might be assessed and the costs.

• Underlined the need to consider the full range of existing approaches to evaluation and assessment for the establishment of the instruments of the feasibility study.

• Noted that countries would base decisions on further steps on the outcomes of the feasibility study.

I would like to thank former Minister of Education, Akito Arima; the President of Tokyo University, Hiroshi Komiyama, and the Director of the William and Flora Hewlett Foundation, Marshall Smith, for their valuable contributions to our discussion. And I am grateful to fellow Ministers for their openness and generosity in contributing to our exchanges which I know will be of great relevance to reform in my own country and will I hope be of equal value to them.