|
The factors affecting higher education are woven so tightly together that they must first be teased apart before an accurate assessment can be made. The AHELO feasibility study thus explores four complementary strands.
Generic Skills Strand
Building a bridge leaves no margin for error. Nor does criminal law, where an oversight could have a devastating effect on a person’s future. Engineers and lawyers have little in common, except that skilled analysis lies at the heart of both their professions.
Analytic reasoning is defined under AHELO as a “generic” skill, applicable to any number fields. Other generic skills include critical thinking, the ability to generate fresh ideas, and the practical application of theory. Ease in written communication, leadership ability, and the ability to work in a group, etc. could also be included. The list is long. The point is that the simple acquisition of knowledge is not enough to count as an education.
Yet there exist few instruments to measure these competencies, and their development would exceed the timeframe of the feasibility study. Thus to determine to what extent generic skills can be measured across diverse institutions, languages and cultures, the feasibility study is adapting the Collegiate Learning Assessment developed by the Council for Aid to Education (CAE) in the United States, to an international context with a view to provide a proof of concept. Students will complete an online assessment, using their critical skills along with data provided for each task. The questions are not specialized so that they can be answered by most undergraduates, whatever their field of study.
Current participants in the Generic Skills Strand: Finland, Korea, Mexico and Norway.
Discipline-specific Strands in Engineering and Economics
Generic skills, however, are rarely a priority for HEIs. Their main goal is to provide students with expertise in various fields and to prepare them for a career. Most HEIs define their missions in these terms. Failure to give due weight to this truism is to overlook what actually goes on in university classrooms. Students and faculty would be astonished if an assessment left out the very reason they are in higher education.
Two strands of the feasibility study measure a student’s competence in his or her discipline. But the simple demonstration of factual knowledge is not enough. AHELO targets the “above content” part of learning: the capacity of student to use what he or she has learned, often in novel circumstances.
Engineering and economics are the disciplines selected for the feasibility study. In the event of a full-scale AHELO programme, the number of disciplines would be increased.
Current participants in the Engineering and Economics Strands:
Engineering: Australia, Japan, Sweden.
Economics: Belgium (Flemish Community), Italy, Mexico, the Netherlands.
The generic skills and discipline strands provide a more refined method of assessment. Whereas rankings stop with the results as they stand, AHELO looks behind the results.
Learning in context
A cold or overcrowded classroom, family pressure to succeed and social taboos that discourage women or certain members of society from seeking a career are just a few of the scenarios in which students and teachers find themselves. Higher education is no fortress, no matter how tenaciously the image of the “ivory tower” holds the imagination. Students come from diverse backgrounds and have different ambitions. What their teachers, peers and potential employers expect of them also differ. Ignoring context distorts any assessment of learning outcomes. Unfortunately, some observers may dismiss context as merely background information, not realizing its crucial importance, leading them to misinterpret results and blame shortcomings on the quality of teaching, curricula, or some other factor.
The AHELO feasibility study examines context through the lens of four topical areas:
-
Physical and organisational characteristics. Observable characteristics such as enrolment figures or the ratio of male to female students.
-
Education-related behaviors and practices. Student-faculty interaction, academic challenge, emphasis on applied work, etc.
-
Psycho-social and cultural attributes. Career expectations of students, parental support, social expectations of HEIs, etc.
-
Behavioral and attitudinal outcomes. Students’ persistence and completion of degrees; continuation into graduate programmes or success in finding a job; student satisfaction, improved self-confidence, and self-reported learning gains claimed by students or their instructors.
Evidence will be gathered from public statistics, previous research, and surveys of students, faculty and HEI administrators. Two additional sources – alumni surveys and employer feedback – will be developed over the long term for future inclusion in a full-scale AHELO.
The Value-Added Strand
Top HEIs draw top students. So it is no surprise when an A+ student walks out the doors as an A+ graduate. But what about a B student who finishes with an A? His or her programme would have a higher added value than the programme at the top university.
Traditional university rankings ignore this aspect. What a student brings to a degree programme and what he or she leaves with are a powerful indicator of teaching quality, availability of resources and the capacity of students to learn.
When pouring over league tables, students want to know the “bottom line” of an HEI: results. And of course good results are crucial. A full-scale AHELO would focus on both types of outcomes: the “bottom line” as well as the “value-added” to reach a balanced picture.
However, measuring the value-added component of an education is complicated. The time required to develop the necessary instruments would exceed the timeframe of the feasibility study. Hence this strand of work will instead reflect on possible methodologies, drawing upon similar work that has already been carried out by the OECD at the secondary education level. Researchers will consider the merits of existing methodologies, and examine psychometric evidence (“psychometrics” is the field of educational and psychological testing) on the basis of existing data collected at the national level.
|
