Abstracts: Institutional measures to assess and improve quality

 

Our Journey Towards World Class: Leading Transformational Strategic Change

Authors: Simon Donoghue, Head of The Strategy Project Group , University of Leeds and Mike Kennerly

 

This paper presents a case study on the development and implementation of institutional strategy within a large, prestigious, research intensive, leading UK University. The strategy process has been approached in a way that uses the University’s international reputation and ranking, alongside its core values and purpose to create a strong platform for change; delivering a World Class institution.

Informed by international best practice, the approach has been recognised internationally as leading edge. The University has used a “strategy map” to provide a framework which links the University’s aspirations in terms of societal impacts (purpose) and reputation and position (vision) to objectives, measures and themes to achieve these strategic outcomes.

The approach incorporates the use of measures to assess each of the strategic outcomes across all parts of the University, enabling academic leaders to balance priorities.  The case study highlights the importance of balancing the academic community’s acceptance of the vision with the mechanisms for assessing progress.

The case study provides a useful insight into how to affect change in a large successful institution improving the quality, reputation and impact it provides.  The University is making significant progress in affecting a substantive strategic change programme.  The paper explores the key elements and themes within this programme ranging from leadership, communication and alignment to cascading key measures of progress, and embedding institutional strategy throughout the academic community. The paper concludes with a number of lessons learned that will be relevant to any HEI looking to affect significant strategic change. 

Assessment reform as a stimulus for quality improvement in university learning and teaching: an Australian case study

Author: Stuart Campbell, Pro Vice Chancellor, Univeristy of Western Sydney

 

Improving student assessment quality and practice is a particular challenge for higher education leaders. Internationally, assessment practice is seen as in need of improvement; at the same time there is a worldwide preoccupation with league tables and standards. A student assessment reform process at the University of Western Sydney is described. This year long process entailed the simultaneous development of a new assessment policy, an assessment guide, and communities of practice around assessment. The reform process had impacts beyond pedagogy: It impinged on management processes, curriculum renewal, attitudes to student centredness, higher education scholarship, governance arrangements, professional development, and industrial relations. In summary, it is argued that student assessment reform is a strong lever for quality improvement in learning and teaching and beyond, and that it poses challenges for higher education leaders in a broad range of management domains.

Defining the role of academics in accountability

Author: Elaine El-Khawas,  Professor of Education Policy, George Washington University

 

While the policy debate on accountability in higher education has been vigorous in many countries, it has focused primarily on broad objectives or institution-level approaches. Limited attention has been paid to the critical role of academics in defining and implementing systems for assessing learning outcomes.  Giving members of the professoriate a central role in accountability is consistent with long-standing principles of academic autonomy. It is also necessary:  implementing accountability meaningfully requires decentralized implementation that recognizes the differing circumstances of study fields and levels. Academics must be involved in a sequence of tasks – developing assessments; testing and refining them against new evidence, making sense of accountability results, and responding with changes in programs or delivery. This paper offers a process model of how members of the professoriate could contribute to the development and use of outcome measures for student learning in higher education.  It identifies ways that professional associations of academics, both nationally and internationally, could develop guidelines for good practice in specific fields (business, education, chemistry, literature, social welfare, etc.). It also suggests approaches within institutions of higher education to adopt such guidelines and to motivate academics to use them to routinely assess student learning outcomes in individual subject areas.

Assessing and Improving Undergraduate Teaching and Learning in the United States: The National Survey of Student Engagement

Author: Alexander C. McCormick, Director, National Survey of Student Engagement, Associate Professor, Indiana University School of Education

 

Since 2000, more than 1,200 four-year colleges and universities in the U.S. and Canada have turned to the National Survey of Student Engagement (NSSE) to assess—in a manner that permits comparisons with peer institutions—the extent to which their undergraduates engage in and are exposed to educational practices associated with high levels of learning and development. “Student engagement” represents two critical features. The first is the amount of time and effort students put into their studies and other educationally purposeful activities. The second is how the institution’s resources, curricula, and other learning opportunities support and promote student experiences that lead to success (e.g., persistence, learning, satisfaction, graduation). The latter feature is of particular interest, as it represents the margin of educational quality that institutions contribute and is something that a college or university can influence.

This paper will first discuss the origins of NSSE and its role in the U.S. assessment and accountability movements, and then move into more practical measurement issues highlighting the conceptual and empirical foundations of student engagement, describing the NSSE instrument, selected findings on engagement and institutional effectiveness, and documenting how institutions use their results to guide improvement efforts. The paper will close with a discussion of current tensions in the U.S. between initiatives to promote internal improvement and external accountability, and the importance of using data responsibly.

Institutional leadership quality and selection: an uncovered factor in the outcomes of Higher Education

Ayodele Fajonyomi, Department of Continuing Education and Extension Services, University of Maiduguri, Nigeria


This is a descriptive, empirical and suggestive paper. It rests on three intertwined axioms. First, higher education (HE) plays momentous roles in personal and societal development. Second, the quality of HE cannot rise above the quality of its leadership, specially at the institutional level. Third, the quality of HE leadership implicates the qualities of input, process and outcome of higher education. While the issue of HE leadership quality is primordial to all issues, it has not been given adequate attention. This neglect explains the dysfunctional state of HE, particularly in Nigeria and probably many other nations in Africa and Asia. In addressing its descriptive purpose, this paper reviews literature on the relationship between leadership quality and HE effectiveness; rehearses the roles of vice-chancellors, provosts, or rectors chief executive officers (CEOs) of HE institutions in providing strategic direction and leadership; and discusses the essential qualities of effective HE leadership. The literature review and discussion of the roles of the CEOs, lead to preliminary report of a study on “The characteristics that make an effective HE leader.” The data were collected from participants at the conference organised by the Nigeria’s education apex body. Consequent on the foregoing, the paper concludes with policy implications for HE development. 

Improving the Success of Students At Risk of Dropping Out: A Report on Ongoing Research Experiments in Canada

Author: Andrew Parkin, Associate Executive Director, Canada Millennium Scholarship Foundation

 

The most important anticipated outcome of entry into post-secondary education is graduation.  Institutions must be successful not only at recruiting students but at seeing students through to the completion of their programs.  As access to higher education continues to expand, institutions must adapt to a more diverse student body, one in which a greater proportion of students may be at risk, for academic, financial or cultural reasons, of abandoning their studies before graduation.  If the quality of the education provided is not to be reduced, and if students are not to be set up for failure, institutions must begin to put appropriate student support programs in place.  To ensure the best use of resources, it is important that the development of such programs be informed by evidence gathered through research.  The Canada Millennium Scholarship Foundation has initiated several research experiments involving the study of the effectiveness of different forms of support.  One of these experiments involves the provision of additional financial aid to low income students and Aboriginal students at various institutions.  Two others, based at three colleges and one university, involve the provision of a mix of academic support, peer mentorship, career counseling, and cultural support for students deemed at risk of dropping out and Aboriginal students.  While final results of the experiments are still some years away, the paper will report on their design and implementation as well as interim findings.  The paper will demonstrate how institutions can use experimental research to inform policy development that will ultimately enhance the quality of the educational experience they offer, thus improving outcomes.

Teaching and learning quality indicators in Australian universities

Author: Denise Chalmers, Professor Carrick Institute for Learning and Teaching in Higher Education

 

This national project to identify and implement teaching and learning quality indicators in universities grew from the recognition that an agreed approach was needed to recognise and reward quality teaching and teachers in higher education.  A key aspect of recognising quality teaching is the development and implementation of agreed indicators and metrics across the Australian university sector.

The first stage of the project involved an extensive international literature review and a survey of practice in Australian universities on the use of teaching and learning indicators at the institutional level.  This paper provides a brief overview of the context in which performance indicators are used in higher education and describes the types of indicators as input, output, process and outcome.  It is argued that all types of indicators are needed if a comprehensive picture of a university is to be obtained. 

A framework of teaching quality dimensions for use at the institutional level has been developed, drawn from empirical research and literature to identify indicators that enhance student learning and learning experiences. This framework is now being trailed in eight Australian universities to assess its usefulness in providing an approach to implement and embed teaching and learning indicators at multiple levels within university, with the view of developing some common indicators that can be used across institutions and disciplines.