Mapping Quality Assurance Indicators for Non-formal Adult Learning

2. Quality frameworks included in the analysis

Abstract

To identify the quality areas and criteria typically included in quality assurance systems for non-formal learning, this report analyses 12 quality assurance systems using the 4 phases of the EQAVET framework and their respective indicative descriptors. This chapter presents the EQAVET framework as well as all quality assurance system included in the analysis, offering an overview of the main characteristics of each quality assurance framework.

The current report relies on the analysis of the EQAVET framework, of ten country-wide quality assurance systems for non-formal adult learning and of ISO quality labels 21001 and 9001. These systems provide a good overview of what the key areas of a quality assurance framework are and reflect a variety of instruments and of quality criteria and indicators. This chapter presents the basic characteristics of each of the quality assurance systems included in the analysis.

The EQAVET framework

The European Quality Assurance Reference Framework for Vocational Education and Training (EQAVET) was established in 2009 through a Recommendation of the European Parliament and of the Council,1 which was updated by and integrated in the 2020 Recommendation on vocational education and training for sustainable competitiveness, social fairness, and resilience.2 This Recommendation provides a common framework for co‑ordinating and assessing the quality of VET delivery across Europe at the system and at the provider-levels (EQAVET, 2023[6]).

The core of the EQAVET framework is the quality cycle, which structures quality assurance processes in four stages: planning, implementation, evaluation and review. These stages include indicative descriptors, both at the system and provider levels, which guide governments and providers on the actions needed to effectively carry out each stage. Completing the whole quality cycle ensures the continuous improvement of the VET system and contributes to developing a VET quality culture in the country and at provider level. In addition, QA indicators are also suggested by the framework. These indicators collect data on the use, delivery and relevance of VET and can be used to evaluate the system and identify potential improvements.

The EQAVET framework was conceived as a “toolbox” from which countries can select the indicative descriptors and indicators that best suit the needs of their quality assurance systems (European Commission, 2023[7]). Thus, EU Member countries decide the extent to which they use the framework and how they implement it. The framework can be either used directly; or applied through existing national QA systems or through third-party quality labels such as ISO 21001. In some countries, like Spain, the full EQAVET framework is adopted, while in others, like the Netherlands, EQAVET is used as a basis to improve their already existing quality assurance system for VET. In other countries, such as Portugal, EQAVET has even been linked to a quality label, the EQAVET label, to indicate the high quality of some providers.

Most European national QA systems for VET are aligned to the EQAVET Framework. About 90% of countries who participated in a survey on the implementation of EQAVET in 2022 reported that their national QA systems comprise the EQAVET quality cycle. While the use of the specific indicative descriptors and indicators is more limited, still over 60% of countries responding to the survey reported using the indicative descriptors and/or EQAVET indicators. Countries making use of the EQAVET framework tend to use its indicative descriptors more for the planning and implementation phases (51% and 44% respectively in the case of CVET) than for the evaluation and review phases (29% and 25% for CVET respectively). However, despite the broad application of EQAVET to formal VET, the use of EQAVET is more limited for CVET than IVET and even lower for adult learning (EQAVET, 2023[6]).

The implementation of the EQAVET Recommendation is supported by a network of National Quality Assurance Reference Points (NRP), who bring together relevant stakeholders to further develop and implement the EQAVET framework. Among the activities carried out by the EQAVET network, NRPs participate in peer learning activities, where good practices are shared, and carry out peer reviews in which they assess another member’s QA system’s features.

Given that the QA systems included in this report mostly define quality assurance criteria and indicators to be used by providers directly, the following chapters include the EQAVET indicative descriptors at the provider level only. The EQAVET indicative descriptors for quality assurance at the system level can be found in Annex B for reference.

Quality assurance systems for non-formal training included in the review

To complement the analysis of the EQAVET framework, ten country-wide quality assurance systems, which cover non-formal training, as well as ISO labels 21001 and 9001 were analysed for this report. As shown in Table 2.1, these include systems based on quality labels and systems based on self-assessments.

Table 2.1. Quality frameworks for non-formal training included in the review by instrument used

Quality Label	Self-Assessment
Ö-Cert, Austria	QQI, Ireland
Cert NÖ, Austria	European Quality Mark (EQM), Iceland
Qualiopi, France	NRTO label, the Netherlands
Label de qualité, Luxembourg	OQEA, Slovenia
DGERT, Portugal
EduQua, Switzerland
ISO quality labels 21001 and 9001

Source: Author’s elaboration.

Basic information for each system, including the number of quality criteria or indicators, is described below.

• Ö-Cert (Austria) is a nationwide quality label for adult education providers that relies on training providers having one of 12 Ö-Cert-approved Quality Management Systems or Quality Assurance Procedures. Since it is based on training providers holding another quality certification, it can be considered an “umbrella label”. It was developed in 2011 by the Federal Ministry of Education and the nine Austrian provinces in co‑operation with relevant stakeholders of adult education, as the Conference of Adult Education Organizations. Between 2012 and 2022, over 550 providers (1 373 including branches) were accredited. Given the requirement that training providers must have one of the 12 external Quality Assurance Systems in place, Ö-Cert itself is very lean, including only 5 basic criteria.

• Cert NÖ (Austria)3 is one of the 12 quality labels accepted by Ö-Cert. Its certification body was set up at Danube University in Krems in 2006, on behalf of the state of Lower Austria. This quality certification includes 10 criteria and, as of March 2023, is used by 102 training providers.

• Qualiopi (France) is a quality certificate that, since 2018, is compulsory for French training centres to receive public funds. It is based on a single national quality reference system (Référentiel national de certification qualité des actions concourant au développement des compétences – RNCQ), which is organised around 7 criteria linked to 32 quality indicators, 22 of which apply to all training providers and 10 of which apply to specific types of providers or to training leading to professional certificates.

• European Quality Mark (EQM, Iceland) is a quality label that was developed jointly with seven European Union partners within the framework of a European Union project between 2006 and 2008. This initial framework was revised in 2014 – 2016, thanks to a grant from the Nordic Council of Ministers Nordplus Voksen educational programme. The label is based on the provider’s self-assessment, which is reviewed by an external assessor, and contains 12 criteria distributed in 3 quality areas. As of February 2022, 18 training providers in Iceland held the EQM.

• Quality and Qualifications Ireland (QQI, Ireland) evaluates the quality of Irish training providers’ programmes and the providers’ quality assurance policies, which, since 2012, must be set up for a provider to operate. Since the QQI framework applies to both, higher and further education systems, QQI’s guidelines are modular. All training providers subject to the system must comply with the core guidelines and, depending on the type of provider (for example, further education and training or independent providers) or the characteristics of the training provided (for example, blended learning) other guidelines may also apply to the provider. The core guidelines include 11 quality assurance areas that providers must take into consideration when designing their quality assurance policies. In addition, after an initial quality review by QQI, training providers must submit annual self-assessments and participate in meetings about quality with QQI specialists. The system applies to about 190 further education providers, which include all 16 Education and Training Boards (ETBs), the further education public providers.

• Label de Qualité (Luxembourg) is a quality label created in 2000 for the non-formal adult training sector in Luxembourg. It contains only 10 criteria that providers of adult training must meet, which mostly concern organisational aspects of the training (e.g. minimum number of participants in each course and special fees for disadvantages groups) or trainer qualifications.

• The General Directorate for Employment and Industrial Relations (DGERT, Portugal) is responsible for certifying the quality of training providers. This certification is based on 24 criteria organised in 3 quality areas (structure and internal organisation, training quality and training results). As of April 2021, 2 780 training providers had been certified by DGERT.

• Offering Quality Education to Adults (OQEA, Slovenia) defines a quality management system for training providers to carry out their own self-evaluations. Since 2018, all adult training providers in Slovenia must have an internal quality system, which must include ongoing monitoring and self-assessment. These quality management systems are based on the OQEA model, which contains 11 quality areas and 98 quality indicators from which providers can choose. Between 2001 and 2013, 69 training providers used the OQEA model.

• EduQua (Switzerland) is a quality label that ensures that Swiss providers of adult learning meet some minimum standards at the time of registering. While it is not compulsory, some of the Swiss cantons require providers to have the EduQua label to receive public funds. It is based on 8 quality areas, which include 19 criteria. As of March 2023, over 1 100 training providers were accredited with the EduQua quality label.

• The NRTO Quality Label (the Netherlands) is a quality label based on a self-assessment carried out by the provider. It was developed by the Dutch Council for Education and Training (NRTO), a trade association of about 300 private training and education providers, and includes eight quality requirements. In addition, 49 indicators classified in 4 quality areas are provided for training institutions to carry out their own self-assessments.

• ISO labels 9001 and 210014 are international quality standards that set out criteria to implement an internal quality management system. ISO 21001 focuses on educational organisations (both formal and non-formal) and includes 11 quality principles. While both ISO quality labels are used and accepted in multiple quality assurance systems, ISO 21001 has been explicitly aligned to the EQAVET framework.

All these quality assurance systems are structured around clusters of indicators or requirements. However, the number of levels of clustering depends on the specific system. For example, while the OQEA model in Slovenia has 5 levels, EduQua in Switzerland or the NRTO quality label in the Netherlands only have 3. The remaining of the report will use four levels of clustering, including quality phase, with each subsequent chapter focusing on a single EQAVET phase in the quality cycle, quality area, quality criteria and indicator or requirement.

To verify training providers’ compliance with the quality assurance indicators, quality assurance frameworks include an assessment method. Typically, they rely on self-assessments, internal or external assessments. The use of each type of assessment is generally correlated with the type of quality assurance instrument used by the specific quality framework (quality label or self-assessment). Table 2.2 summarises how the different QA systems analysed for this report assess and monitor compliance with their indicators, including the fees that training providers must cover to participate in the quality assurance framework.

Table 2.2. Overview of quality framework characteristics with regard to the application and assessment processes

Quality framework	Quality instrument	Initial external assessment	Initial on-site audit	Self-evaluation	Intermediate external monitoring	Duration of validity	Price
Ö-Cert (Austria)	Quality label	Yes	No (the quality label in which the Ö-Cert award is based on must include an on-site visit)	No	No	The validity depends on the validity of the quality label the Ö-Cert award is based on (maximum 4 years).	EUR 100 for initial certification and EUR 100 for renewal. This excludes the cost of the quality label the Ö-Cert label is based on.
Cert NÖ (Austria)	Quality label	Yes	Yes	No	No	3 years	EUR 1 899 for initial certification + EUR  450 for initial audit.
Qualiopi (France)	Quality label	Yes	Yes	No	One year after award of quality label	3 years	EUR 1 200‑1 700 pre‑tax, depending on the size of the provider, the number of sites and the number of programmes.
EQM (Iceland)	Self-assessment	Yes	Yes	Yes	Training providers submit annual self-assessments	3 years	An annual fee of about EUR 1 550.
QQI (Ireland)	Self-assessment	Yes	Yes	Yes	Training providers submit annual self-assessments and participate in quality dialogue meetings with QQI on a periodic basis. In-depth external review every 5‑7 years. QQI validation of training providers’ programmes, which must be revalidated every 5 years.	As long as the provider satisfies the criteria.	EUR 5 000 for new providers of Further Education and Training (FET) programmes for the submission of quality assurance procedures for approval. Additionally EUR 1 000‑2 000 in fees for validation of Further Education and Training programmes.
Label de qualité (Luxembourg)	Quality label	Yes	No	No	Not specified	Not specified	Not specified
NRTO (the Netherlands)	Self-assessment	Yes	Yes	Yes	Training providers submit a quality declaration stating that they meet the NRTO criteria annually.	4 years	Around EUR 1 000 for the initial external audit, the actual cost depends on the auditing institution. In addition, an annual fee of EUR 50‑350 to NRTO.
DGERT (Portugal)	Quality label	Yes	If deemed necessary	No	Yearly documental monitoring that can lead to an external audit.	As long as the provider satisfies the criteria.	EUR 500 up to 3 training or educational areas then EUR 150 for each additional area.
OQEA (Slovenia)	Self-assessment	Yes, to check whether providers carry out self-assessment	No	Yes	No	As long as the provider satisfies the criteria.	Not specified
EduQua (Switzerland)	Quality label	Yes	Yes	Yes	Yearly intermediate audits	3 years	For providers with less than 25 000 trainees, about EUR 5 000 excluding taxes and fees.
ISO 9001 and 21001	Quality label	Yes	Yes	Yes	Yearly intermediate audits	3 years	On a case‑by-case basis

Source: Author’s elaboration.

Notes

← 1.  The Recommendation of the European Parliament and of the Council of 18 June 2009 on the establishment of a European Quality Assurance Reference Framework for Vocational Education and Training can be accessed at the following link: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32009H0708(01)&qid=1611571795661.

← 2.  The Council Recommendation of 24 November 2020 on vocational education and training (VET) for sustainable competitiveness, social fairness and resilience 2020/C 417/01 is available at the following link: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32020H1202(01)

← 3.  For the rest of the report, Cert NÖ will be analysed in conjunction with Ö-Cert, as it is one of the quality labels that training providers must hold to be eligible to receive the Ö-Cert quality label.

← 4.  ISO labels 9001 and 21001 will be analysed jointly in the rest of the report, given the similarity in their criteria.