While virtually all formal adult education programmes have some sort of quality assurance mechanisms, quality assurance in non-formal training is rare and more scattered. In most countries, the sector defines its own professional standards (organic approach). However, following some internal or external pressures, the authorities may decide to intervene, thereby facing the choice between imposing minimum quality requirements for providers to operate or access public funds (regulatory approach) and just advising providers on how to best improve their quality (advisory approach). Two sets of tools are typically used to demonstrate quality: quality certificates and labels, and (self-)evaluations.
Improving the Quality of Non‑Formal Adult Learning
Learning from European Best Practices on Quality Assurance
Abstract
Given the abstract nature of the term “quality”, definitions are scattered throughout the education literature. To promote a common understanding, the European Centre for the Development of Vocational Training (Cedefop) of the European Union created a glossary in 2011 on the terminologies used for quality in education and training. Quality is defined as “all characteristics of an entity that bear on its ability to satisfy stated and implied needs” or “the degree to which a set of inherent characteristics fulfils requirements” (Cedefop, 2011[1]). The safeguard of quality has been labelled quality assurance, that represents all “activities involving planning, implementation, evaluation, reporting, and quality improvement, implemented to ensure that all education and training (content of programmes, curricula, assessment and validation of learning outcomes, etc.) meet the quality requirements expected by stakeholders” (Cedefop, 2011[1]). Although related, quality assurance and quality control do not coincide and should not be confused: quality assurance focuses on making sure that the processes to achieve certain results are of high quality, while quality control focuses on the end result itself.
Despite being often characterised by different sector-based regimes, all formal adult education programmes generally have a quality component (Broek and Buiskool, 2013[2]), be it in the form of self-evaluations, external evaluations, or the adoption of existing quality systems (such as ISO standards). Furthermore, the European Union and its institutions have been quite active in developing quality standards and guidelines in higher education and vocation training during the past few years. For example, in 2009 the European Parliament approved the European Quality Assurance Reference Framework for VET (EQAVET) in order to ensure quality of VET provision, while the Standards and Guidelines for Quality Assurance in the European Higher Education Area have been adopted in 2005 and recently revised in 2015 (for more details on the European Union approaches to quality assurance in continuing adult education see Box 1.1).
In contrast, defining in a standardised way the concept of quality in non-formal adult education has proved to be very challenging – let alone assuring quality throughout the whole sector. As a result, quality assurance mechanisms in non-formal training differ from country to country, and how quality is achieved and monitored can be very different (Prisăcariu, 2014[3]). Very often, countries do not have any national-level quality framework in place for non-formal training. When they do, these quality frameworks typically emanate from bottom-up initiatives by the providers themselves.
Why is it harder for governments to set up quality assurance mechanisms for non-formal training? Part of the explanation lies in the costs (both financial and not) of such mechanisms. In fact, quality assurance often requires both financial and human resources that providers of non-formal training might not have. Compared to formal education, the time spent on course planning and implementation in non-formal training is also much shorter or varied, making it sometimes challenging to adopt quality assurance mechanisms. Moreover, while by its very nature formal education fully embraces inspections and compliance with rules, non-formal training providers tend to shy away from bureaucracy (Latchem, 2012[4]). There might also be some resistance from formal operators in allowing a more formalised certification of the quality of their non-formal and informal competitors.
Another significant obstacle to the development of a national quality assurance framework in non-formal adult training is the fragmentation of the numerous different approaches to quality of the sector itself (European Commission, 2013[5]). Such a diverse landscape makes it difficult to stimulate quality development, especially in countries where governance is highly decentralised. In fact, whilst in several European countries policies for adult learning are set at national level, responsibilities can be decentralised to regional and local levels, as is the case for example in Austria, France, Denmark, Sweden, Norway and the Netherlands. Other important challenges to the development of quality assurance in non-formal adult education are linked to the lack of monitoring data essential to understand the participation and learning outcomes of students, and the limited availability of validation of prior learning and lifelong guidance, which are central to the quality of adult education since they enable access, participation and progression.
In spite of these challenges, several European countries have put in place initiatives to improve the quality of their non-formal adult education sectors. Adapting the framework by Hooley and Rice (2019[6]), from a theoretical point of view the various efforts can be grouped into three approaches: (1) the regulatory approach typically sets out clear minimum quality requirements that providers need to meet in order to be recognised; (2) the advisory approach advises providers on what quality should look like, providing guidelines and examples of good practices for them to follow; and (3) the organic approach leaves to providers the overall definition of their own professional standards and quality systems. Before going into details on each framework, it is important to stress that, while theoretically different, these three approaches to quality assurance are not always clear-cut and they can co-exist in some countries.
In practice, quality assurance in all countries starts – either deliberately or unintentionally by not taking any explicit stance on the matter – from an organic approach, allowing providers to completely self-manage their quality without even providing a common definition of quality in adult education. At a certain point, internal demands by policy makers or external pressures from, for example, civil society and NGOs may call for the harmonization and steady development of the quality of training provision across the whole sector. At that point, public authorities face the choice between imposing certain minimum quality requirements for providers to operate or access public funds (the regulatory approach) and just advising providers on how to best improve their quality (the advisory approach). Clearly, the trade-off between these two paradigms is that the latter leaves more room for providers to best find their own paths to quality while the former allows ensuring a minimum, more uniform level of quality across the sector.
The choice between the two approaches typically depends on the motivations that induced the authorities to develop a quality assurance mechanism in first place – with stronger pressures typically leading to the stricter regulatory approach. In addition, there are other two reasons for why, over past years, certain countries have preferred non-binding evaluations to binding certifications. On the one hand, the adult training market may not be mature enough to be subject to external quality requirements, which, if enforced, would probably crowd out most of the existing providers. On the other hand, in certain contexts there may be no need to impose quality disclosure through certifications since organisations may have incentives to voluntarily disclose quality. This is the case, for example, when most providers in the market consider their services to be of good quality and henceforth willingly publicize their quality information. In these circumstances, all providers are compelled to disclose their quality since, in the absence of disclosed information, consumers could infer that the organisation is concealing poor quality (Grossman, 1981[7]). In both cases – the training market not mature enough or widespread voluntary disclosure of quality information – self-evaluations are the typical tool to signal quality in advisory approaches.
Once countries have decided to move away from an organic approach and have made the choice between following the regulatory or advisory approach, they need to select the appropriate tools to foster a quality culture (Figure 1.1). The most used tools throughout the OECD area are certifications and quality labels, evaluations by an external body and self-evaluations by providers. Furthermore, countries have put in place a whole plethora of additional support structures aimed at helping non-formal training institutions improve their overall quality.
Certification is the “process by which a third party gives written assurance that a product, process or service conforms to specified requirements” (Cedefop, 2011[1]), and aim at attesting that relevant quality actions are conducted.1 Providers of adult training need to meet certain minimum quality standards in order to be certified, thereby ensuring that customers are offered effective and efficient training. In addition, other strengths of the certification framework include the fact that it represents an evaluation tool to providers themselves, as well as a monitoring tool for policy makers. To indicate that compliance with standards has been verified, countries also often rely on quality labels. Since labels are a form of communication targeted directly to the end consumer, to be effective and meaningful they not only need to be backed up by a good certification system without conflicts of interest, but the system must also be transparent, information on the content and the organisation behind the label must be accessible, and the meaning of the label must be consistent across all bodies carrying it. In light of such transparency, many European countries have made certification and quality labels compulsory in order to receive public funding.2 As it appears clear, certifications and quality labels are used only in the regulatory approach.
At the opposite side of the spectrum are self-evaluations by providers. Based on public, general guidelines, national and international best practices, and common standards of the sector, this tool helps providers develop their quality by self-assessing the current value of their training services and setting up a plan to improve it in the near future. Self-evaluations are commonly used by countries adopting an advisory approach to quality assurance, since they allow for a great flexibility and low levels of external control (but not always, as is the case in few European countries, where self-evaluations is compulsory in order to receive public funds). In contrast, evaluations by external bodies lie somewhere in the middle. In fact, they are typically based on precise guidelines, but they do not entail fixed minimum quality requirements and targets, thereby leaving more discretion to the evaluation body to assess providers’ quality levels. External evaluations are usually tied to rights to operate or access to public funding (like in the regulatory approach). Although seemingly less demanding than certifications and quality labels for providers, the evaluation of the quality of training programmes and providers can be a challenging task, as evaluation exercises require information on many different aspects. Effectiveness of training is generally measured by looking at training outcomes, such as labour market entry, or satisfaction with the provided training. These outcomes can be assessed through a variety of monitoring and evaluation methods, including audits, on-site inspections, and reports.
To strengthen the common European labour market and build equitable, sustainable and knowledge-based societies, over the past two decades a series of recommendations and initiatives have been developed at the EU level to help member countries move towards a model of quality adult training. This box briefly reviews the major milestones from 2000 to nowadays (Table 1.1).
At the turn of the millennium, the so-called Lisbon Strategy 2000‑10 was one of the seminal EU initiatives recommending greater investments in adult learning with the ultimate goal of upskilling and improving both economic development and social inclusion. At the same time, the European Commission funded the European Forum on Quality in VET as a platform for collaboration between Member countries, the social partners and the European Commission in the area of quality assurance in vocational education and training (VET). The forum developed a work programme for 2001‑02, focusing on four central areas: (1) quality management approaches for VET providers; (2) self-assessment in VET institutions; (3) types of examination and certification practices; and (4) indicators for a European quality in VET strategy.
To facilitate access to lifelong learning, the 2002 Copenhagen Declaration acknowledged the importance of transparency, comparability, transferability, and recognition of competences and qualifications between different countries. The development of reference levels, common principles for certification, and common measures, including a credit transfer system for vocational education and training, was recommended. In this context, the Common Quality Assurance Framework (CQAF) for vocational education and training was developed in 2004. CQAF was a quality management framework based on best practices from Member States and aimed at serving as a reference for the development of quality in national VET systems by describing basic principles, criteria and instruments for the implementation of quality assurance processes.
The following year, the European Commission established the European Network on Quality Assurance in VET (ENQAVET) in order to provide a sustainable platform to support the implementation of the Copenhagen Declaration. In particular, ENQAVET attempted to develop a culture of quality assurance and continuous improvements across the EU and common guidelines for the development of quality assurance in VET systems.
Building on these initial EU initiatives, in 2008 the European Parliament and the Council of the European Union adopted a recommendation establishing the European Qualifications Framework (EQF), a framework encompassing all education and training qualifications. All member states were encourage to link their national qualification systems to the EQF and create National Qualifications Frameworks (NQF) in order to simplify comparisons across countries.
At the same time, the European Commission put forward an Action Plan for Adult Learning “It is always a good time to learn” 2008‑10, which formulates five priorities for the adult learning sector: (1) analyse the effects of EQF, NQF and quality assurance systems reforms in all sectors of education and training in Member States on adult learning; (2) improve the quality of provisions in the adult learning sector, with a specific focus on the initial and continuing training of adult learning staff, quality standards and the accreditation of providers; (3) increase the possibilities for adults to go “one step up” and achieve a qualification at least one level higher than what they currently have; (4) speed up the process of assessment of skills and social competences and have them validated and recognised in terms of learning outcomes; and (5) improve the monitoring of adult learning sector, stressing the need for a common language, indicators and benchmarks, and comparable core data.
The 2010 Bruges Communiqué defined the priorities for the VET sector to 2020, prominently including the promotion of flexible pathways between the VET sector, general education, and higher education and the establishment of comprehensive national qualification frameworks based on learning outcomes (Dollhausen et al., 2013[8]). The Communiqué highlighted the creation of a European Quality Assurance Reference Framework for Vocational Education and Training (EQARF) and a European Credit System for Vocational Education and Training (ECVET) and setting deadlines for their implementation. ECVET allows learners to accumulate and transfer their learning in units, enabling learners to build a qualification at their own pace from learning outcomes acquired in both formal, non-formal and informal contexts. EQARF complements the work on quality assurance of the European Qualification Framework (EQF) and builds on the earlier Common Quality Assurance Framework (CQAF), providing a European-wide system to help countries monitor, evaluate and improve the effectiveness of their VET provision and quality management practices. The same year, the EQAVET network was established to replace the former ENQAVET platform, with the objective of encouraging and supporting the national implementation of the EQARF.
As a follow-up to the 2008‑10 Action Plan, in 2011 the Council of the European Union approved a resolution on a renewed European Agenda for Adult Learning (2011/C 372/01) for the 2012‑14 period. Among other recommendations, this document called for: raising motivation for participation; information and guidance systems; second-chance opportunities; flexible learning pathways; quality assurance systems and accreditation systems; and adult education staff training systems (Antunes, 2019[9]). As a consequence of these recommendations, a thematic working group (TWG) on quality in adult learning (comprising 19 Member States and 2 non-EU states) was established with the mandate of elaborating recommendations for the European Commission and the Member States on the development of quality assurance systems. Their final report came out at the end of 2013.
Finally, on 19 December 2016 the Council of the European Union adopted the Recommendation “Upskilling Pathways: New Opportunities for Adults”, which made provision for a three‑step mechanism focusing on skills assessment, provision of a tailored, flexible and quality learning offer, and validation and recognition of skills acquired. The Recommendation provides that, where possible, within one year of its adoption and at the latest by mid‑2018, Member States should have outlined appropriate measures for the implementation at national level. Based on information provided by the Member States, the European Commission published in February 2019 a report taking stock of their implementation progress and showing that more efforts are required from Member States if they are to achieve the objectives of the Recommendation (European Commission, 2019[10]).
References
[9] Antunes, F. (2019), “Europeanisation and adult education: between political centrality and fragility”, Studies in Continuing Education, pp. 1-18, http://dx.doi.org/10.1080/0158037X.2019.1615425.
[2] Broek, S. and B. Buiskool (2013), Developing the adult learning sector: Quality in the Adult Learning Sector.
[1] Cedefop (2011), Glossary: Quality in education and training, Cedefop, Luxembourg, http://dx.doi.org/10.2801/94487.
[8] Dollhausen, K. et al. (2013), Developing the adult learning sector - Opening higher education to adults.
[10] European Commission (2019), Council Recommendation on Upskilling Pathways: New Opportunities for Adults Taking stock of implementation measures.
[5] European Commission (2013), Thematic Working Group on Quality in Adult Learning Final Report.
[7] Grossman, S. (1981), “An Introduction to the Theory of Rational Expectations Under Asymmetric Information”, The Review of Economic Studies, Vol. 48/4, p. 541, http://dx.doi.org/10.2307/2297195.
[6] Hooley, T. and S. Rice (2019), “Ensuring quality in career guidance: a critical review”, British Journal of Guidance & Counselling, Vol. 47/4, pp. 472-486, http://dx.doi.org/10.1080/03069885.2018.1480012.
[4] Latchem, C. (2012), Quality Assurance Toolkit for Open and Distance Non-Formal Education.
[11] OECD (2019), Individual Learning Accounts : Panacea or Pandora’s Box?, OECD Publishing, Paris, https://dx.doi.org/10.1787/203b21a8-en.
[3] Prisăcariu, A. (2014), “Approaches of Quality Assurance Models on Adult Education Provisions”, Procedia - Social and Behavioral Sciences, Vol. 142, pp. 133-139, http://dx.doi.org/10.1016/j.sbspro.2014.07.623.
← 1. While the terms “certification” and “accreditation” are often used interchangeably, they refer to two distinct processes (OECD, 2019[11]). Certifications guarantee that the necessary steps to achieve a certain level of compliance have been completed. Accreditation procedures, instead, are “the formal recognition by an appropriate authority that a body or a person is competent to carry out specific tasks” (Cedefop, 2011[1]), and they therefore guarantee that the bodies producing a certification are conform and competent to do so. Thus, accreditation and certification do not intervene at the same level: certification is delivered by certification bodies, while accreditation is delivered by accreditation bodies in charge to assess these certification bodies.
← 2. Throughout the remainder of the report, the terms “certification” and “quality label” are used interchangeably.