Quality labels and certificates are typically part of the regulatory approach to quality development in adult learning as formal recognition by external bodies that training providers meet certain predetermined minimum quality requirements. They are used extensively throughout Europe with many successful experiences. Some of these quality labels exist since over 20 years, proving how effective they are in maintaining the quality of adult training. Although most certification processes follow similar practices, they remain very diverse in their implementation and in the quality criteria behind them.
Improving the Quality of Non‑Formal Adult Learning
Learning from European Best Practices on Quality Assurance
Abstract
The eduQua certification framework in Switzerland
One of the best-known and long-standing certification frameworks in Europe is Switzerland’s eduQua. At the turn of the millennium, the adult education sector in Switzerland was highly heterogeneous, with the market for adult learning dominated by many small private providers and no nation-wide regulation (in the Swiss federal system, responsibility for education lies with the 26 cantons). As a consequence, in 2000 the Swiss Federation for Adult Learning (SVEB) – an umbrella non-governmental organisation representing both public and private institutions, associations, and personnel managers – decided to introduce the quality label eduQua, with the support of the State Secretary for Economic Affairs, the cantons, and the Swiss Association of Employment Departments.
The main goal of the certification is to ensure that providers of adult learning meet some minimum standards at the time of registering (details on these minimum standards are presented later in this report). The certification process involves documentation but also on-site visits and yearly intermediate audits. All providers of adult education can apply for the certification, including those involved in the “re-skilling” of unemployed people. An important feature of the programme is that the eduQua label certifies the whole institution, and not its individual courses. Seven eduQua agencies conduct the certification, and over 1 000 Swiss adult learning providers are currently certified. The certification lasts three years, after which the provider must undergo a renewal. Each canton can choose whether providers need to have the eduQua certification to receive public funds, and at the moment this is the case in almost half of the cantons.
The involvement of the public sector in the creation and establishment of the label has been key not only in order to provide the necessary resources and funds in the initial phase, but also to ensure the involvement of all the different stakeholders. As a direct consequence of this, an Advisory Group was established at the outset, whose role is to identify quality criteria behind the label and decide on the requirements for the audits. The Group was originally chaired by the State Secretary and composed of an equal number of representatives from the state and from the different cantons as well as important players in the continuing education landscape (including public and private training providers from various sizes). The inclusive nature of the Advisory Group have proved crucial to ensure strong engagement of the different stakeholders in the initiative. Only after the label had been in place for 18 years and was mature enough to stand on its own feet, the State Secretary for Economic Affairs left the partnership.
The establishment of the eduQua quality label was also strongly supported by providers in an attempt to increase the perceived value of their training courses, and remain competitive vis-à-vis other education centres. This emerges clearly from the results of a survey conducted in 2017, where a majority of certified training institutions stated that they applied to the eduQua label because the whole procedure increases their recognition and credibility in the education market. The other two top motivations to apply for the certification flagged by respondents are also very important: providers see real improvements in the quality of their offer thanks to the certification process; and, in many cases, the label is a precondition for public funding.
With its two decades of experience, eduQua has proved to be very successful in signalling quality of training provision. Its main strengths lie in the fact that its scope is well defined, through a clear objective of certifying the quality of methodology and didactics in adult training only (rather than the whole education sector), and that it is managed by a well-respected main actor (Swiss Federation for Adult Learning, SVEB), which represents the interest of all stakeholders involved. The eduQua initiative itself had been regularly evaluated, allowing for incremental improvements that have increased its effectiveness over time. As a result, the label has received wide support from both policy makers, educational institutions and the private sector over the years.
The Greta-Plus, Eduform and Qualiopi labels in France
The Greta-Plus label in France provides some useful lessons on the importance of securing the buy-in of all relevant partners. The label was created in 2001 by the Ministry of Education with the aim of promoting the Greta (groupement d’établissements publics locaux d’enseignement, i.e. groups of public local providers of adult learning) in the face of private sector competition. The label emphasised the pedagogical dimension of training and the need to provide adults with tailor-made instruction whereby the learning paths and modalities are individualised. Whilst not compulsory, the government promoted the certification as a means of quality assurance. Yet, by 2013, more than a decade after the creation of the Greta-Plus label, only 43 of the 137 Greta had acquired the label (France Strategie, 2013[1]). This disappointing outcome was partly due a limited direct relevance to labour market needs. Indeed, according to Broek and Buiskool (2013[2]), local businesses – the most important “customers” of the Greta – saw little value in this label, thereby limiting Greta’s incentives to apply for it. Moreover, it has also been argued that some Greta preferred not to request a Greta-Plus label in order to avoid the involvement of the central government in their functioning.
As a result, in 2017, the Greta-Plus label was discontinued and replaced by the new Eduform label. Also developed by the Ministry of Education, this new certification system promoting quality in the adult learning sector is applicable to both public and private adult training organisations. The Eduform label has the twofold purpose of boosting centres’ attractiveness and quality by guaranteeing compliance with the AFNOR standards of continuous training services (AFNOR BP X50‑762).1 It is issued for three years after the completion of a national audit, although each year a follow-up audit is organised to review the attribution of the label. The National Council for Employment, Training and Vocational Guidance (CNEFOP) has included Eduform in the list of certificates and labels eligible for the Personal Training Account (CPF), meaning that prospective learners can use the balance on their Personal Training Account to take up courses in institutions with the Eduform label.
However, Eduform is not the only quality label available in France to certify providers of adult learning. In 2018, CNEFOP listed 32 recognised certificates.2 Few of them are issued by public bodies (such as Eduform by the Ministry of Education or the Certif’région label by the Occitanie region), while the majority are produced by private entities (such as the Cequaform label by BCS Certification or Qualiformapro by Dekra Certification). Academic research suggests that, while not inherently problematic, the coexistence of multiple quality labels may lead to confusion among both providers, that are uncertain about which certificate to pursue, and prospective learners, who do not know which label to trust (Banerjee and Solomon, 2003[3]). Moreover, it might also lead to a race to the bottom, with providers applying for the most lenient certification among the ones available (Cashore, Auld and Newsom, 2003[4]).
In order to simplify this complex quality assurance landscape, in September 2018 the French Government passed a law establishing that all training centres that wish to obtain public funds must obtain a new quality certificate – Qualiopi – based on a single national quality reference system (“Référentiel national de certification qualité des actions concourant au développement des compétences” – RNCQ).3 Providers can freely choose their certification body – as long as it is a body accredited by the French Accreditation Committee (COFRAC) – and must be certified by 1 January 2022 if they wish to receive public financing.4 One of its peculiarities is that Qualiopi audits play a central role. In fact, not only do the certifying bodies carry out an on-site audit during the initial application process to ensure that what is self-reported by providers is properly implemented and corresponds to the RNCQ, but inspections take place one year later. If the certification body detects non-compliance, the label may be suspended or withdrawn. Moreover, when the Qualiopi label expires after three years, a new on-site audit is carried out to renew the certification for another three years.5
The certification of providers in Germany
The adult learning sector in Germany is less regulated by the state than other areas of education, under the assumption that local providers can more easily meet the diverse and rapidly changing demands of adult learners. Yet, over the years, there was a drive towards the certification of non-formal learning as an incentive for adults to engage more fully in society, leading to the elaboration of a nationwide certification process for all providers offering measures of active employment promotion in 2012. Under the process, providers have to be certified by specific bodies (Fachkundige Stellen) if they want to provide training. The German Accreditation Body (Deutsche Akkreditierungsstelle) is in charge of accrediting the certification bodies to make sure certification standards and procedures are adequate. The certification can be granted for a maximum of five years. If certified, providers can benefit from public funding.
Irrespective of the certifying body, the certification procedure for providers is structured in three stages. The first step involves the approval of the provider, and is mandatory for all; the prerequisites are efficiency and reliability, personnel and technical suitability, appropriate contractual conditions for the participants, and a quality assurance system (such as external quality management systems). In particular, this latter system for quality assurance must include requirements on: (i) customer orientation; (ii) continuous evaluation of training courses based on the use of indicators and measurement; (iii) continuous improvement of training provision; and (iv) cooperation with external experts for quality development. The second step involves the approval of the courses. For this, the certification body checks whether the course concept is likely to lead to successful completion, whether it is expedient, economical and whether it offers appropriate conditions for participation. The third stage is only necessary for providers of continuing vocational training and includes additional requirements for them.
The Initiative for Adult Education and the Ö-Cert quality label in Austria
In Austria, the adult education landscape is characterised by few national regulations and a large variety of stakeholders. No unified quality assurance system for non-formal adult learning exists, but a range of different instruments has been put in place over the past decade to ensure quality education. Two initiatives, in particular, have drawn great attention in the international debate on quality assurance: the Initiative for Adult Education and the label Ö-Cert.
The Initiative for Adult Education (Initiative Erwachsenenbildung) was established in 2012 by the Federal Ministry of Education together with the nine Austrian provinces, with the goal of creating high-quality courses enabling low-skilled adults to continue – and, in many cases, finish – their education. In addition to providing free courses for all participants, an important feature of the project is the implementation of consistent quality guidelines for all courses in the country. Accreditation is necessary for providers to take part in the initiative and it is based on three quality criteria: (1) fulfilment of general requirements; (2) creation of an appropriate programme concept; and (3) fulfilment of the project guidelines concerning the qualification of the trainers and counsellors. Once accredited, the provider can apply for public funding. Typically, for an approval to be granted, not only should the quality guidelines be met, but the programme should also fit with the needs of the participants in the region where it is conducted – in other words, funds are only granted when there is a need and a target group for the accredited offer. Providers taking part in the Initiative commit themselves to continuous monitoring and evaluation. This whole accreditation process is undertaken by six selected adult education experts, and a monitoring board supervises the process and the results.
In contrast, Ö-Cert is a quality framework (“umbrella label”) for all adult education providers. Developed in 2011 by the Federal Ministry of Education and the nine Austrian provinces in cooperation with important stakeholders of adult education – such as the Conference of Adult Education Organizations – the aim of the label Ö-Cert was to reduce the administrative burdens that both providers, prospective learners and public authorities used to encounter in identifying quality training. In fact, depending on the definition used, there were between 1 800 and 3 000 providers in Austria in 2007, with offers that were often difficult to compare and little transparency for customers (Gruber, Brünner and Huss, 2009[5]). With rising attention on quality issues, providers had also gradually started to adopt a multitude of quality labels, systems and seals (e.g. ISO, EFQM, LQW, …), making it so difficult for the government to assess their quality when applying for public funding that even provinces had started to create their own quality controls. The introduction of a single quality label was thus important not only to encourage homogeneity in quality assurance, but also to better manage the sector.
At its inception, Ö-Cert therefore had a challenging task: ensuring quality across educational institutions without being an additional quality management system. To address this, Ö-Cert not only committed to safeguard the principle of autonomy of providers, but – to not overburden all actors involved with a new quality management system – it pledged to be only a system of recognition and certification of quality without imposing their own audits. In practice, in order to be accredited with the Ö-Cert label, providers only need to have one of 11 Ö-Cert-approved Quality Management Systems or Quality Assurance Procedures. Hence, by using the concept of “umbrella label”, this top-down procedure manages to respect the autonomy of decisions and use of different quality assurance systems by providers, while at the same time being an effective, simple and cheap model of introducing a transparent tool of quality in diverse contexts of the adult learning sector (Broek and Buiskool, 2013[2]). Since 2012, over 460 providers (1 266 including branches) have been accredited.
The NRTO and KET-KIT quality labels in the Netherlands
Similarly to Austria, in the Netherlands the adult education sector is also made up of numerous stakeholders and providers. While so far a nation-wide quality assurance system does not exist (but it is about to be created, see the last section of this chapter), in the past few years some quality initiatives have been put in place for a smaller subset of providers. For example, the Dutch Council for Education and Training (Nederlandse Raad voor Training en Opleiding, NRTO) is an umbrella trade association of about 300 private training and education providers, which established its own quality label. The NRTO quality label is based on an organisation’s self-evaluation that is validated by an external auditor. The external audit is conducted by one of the three recognised organisations (Kiwa Nederland, CPION, CIIO) and costs around EUR 900. The basis of the NRTO quality label consists of eight quality requirements that are important for every private provider, from classroom trainers to providers of (online) training courses and exam and validation institutes. An NRTO member:
Is transparent about its product or service;
Is clear about the learning outcomes of education and training;
Measures customer satisfaction;
Fulfils agreements made;
Uses knowledgeable teachers, trainers and advisors;
Invests in the expertise of its staff;
Has its processes in order;
Strives for continuous improvement.
Like NRTO, three large language training organisations in the Netherlands – ITTA, Radboud in’to Languages and VU NT2 – have established their own instrument to ensure quality in language education: KET-KIT (Kwaliteitsinstrument Taalonderwijs van de Kwaliteitsgroep Educatie Taal). KET-KIT distinguishes five quality areas (didactics, student guidance, facilities, management, and quality assurance) and within that a number of quality indicators and specific criteria, which have to be used to prepare a self-evaluation and which are then verified during an onsite inspection. If the self-evaluation and the audit provide evidence of sufficient quality, the organisation is given the KET-KIT quality label for two years. The standard price for KET-KIT is EUR 5 600.
The GKK quality framework in Belgium
In Flanders (Belgium), a distinction is made between adult education and adult training. On the one hand, adult education comprises all formal programmes of adult basic education, secondary adult education and adult higher vocational education. These courses are provided by the so-called Adult Education Centres and the Adult Basic Education Centres. On the other hand, adult training is a broader concept that includes, next to formal programmes, all forms of non-formal learning by adults, including the programmes offered by the Flemish Employment Services and Vocational Training Agency, socio-cultural organisations, private institutions, etc. While the Centres for Adult Basic Education and the Centres for Adult Education are required to implement a quality assurance system, until recently providers of non-formal training did not have a common quality assurance framework.
Only in June 2018 did the Flemish Government approve a preliminary draft by the Department of Education of a new common quality framework (Gemeenschappelijk kwaliteitskader, GKK) that aims at evaluating the quality of VET training provided by institutions outside the formal education system. The framework is based on the Flemish Qualification standards and is in line with the European Qualification Framework (OECD, 2019[6]). The proposal has been transformed into a decree in April 2019.6 The new legislation states that non-formal or informal organisations that wish to offer recognised vocational programmes leading to a qualification that is part of the Flemish Qualification Structure must first apply for accreditation. As part of the process, a quality check is performed every six years, including on-site visits, by a neutral and independent external supervisory body. The framework includes the following quality areas:
The objectives of the training correspond to the competences required by the relevant professional qualification;
The design of the training is elaborated and organised in such a way that learners can acquire or demonstrate the competences required by the relevant professional qualification;
The guidance of the learners implies that they are offered optimal opportunities to acquire or demonstrate the competences required by the relevant professional qualification;
The evaluation of the learners allows to verify that they have acquired the competences required by the relevant professional qualification;
the action points established in connection with the objectives, design, guidance and evaluation of the training lead to real quality improvements.
Training providers in Korea wishing to deliver government-funded training programmes need to be certified. The duration for which certification is granted depends on the outcome of the quality evaluation. The Korean Skills Quality Authority (KSQA) is in charge of the evaluation of vocational training providers, training programmes and trainees. The KSQA conducts an in-depth evaluation of institutions, including on financial soundness, capacity to provide training and training performance, and grants certified grades based on the evaluation outcomes. These grades are necessary to access government-funded training, and better performing institutions receive grades that are valid for longer periods (up to five years). The KSQA also screens training programmes in terms of content, methods, teacher quality, facilities and equipment, and past training outcomes. For the evaluation of the trainees, the KSQA assesses whether the participants who completed training courses have acquired the expected skills. Courses that have positive outcomes regarding trainee evaluation can receive additional financial support. The results from the trainee evaluation also feed into the training providers’ evaluation.
In Chile, providers of training financed by the public employment services have to adhere to a Quality Norm that was set in 2015. Certification based on this Quality Norm is done by private entities (Organismos certificadores de servicios), which in turn are supervised by a public entity (Instituto Nacional de Normas). The aim of the Norm is to ensure that providers’ management prioritise the satisfaction of the participants in the training activities and invest in the development of trainers’ skills. The Norm also aims at periodically generate information on providers’ financial results. When the Norm started to be enforced in 2017, this led to the closure of around 800 training providers. Complementary quality control mechanisms include ex-ante evaluations of training courses, evaluation of teaching staff (where studies, teaching and work experience of course trainers are evaluated), and on-site audit processes.
Source: OECD (2019[7]), Getting Skills Right: Future-Ready Adult Learning Systems, https://dx.doi.org/10.1787/9789264311756-en.
International quality standards
In addition to national certification frameworks, some international standards have also entered recently the market of quality assurance labels of adult learning. In particular, the most widely known are issued by the International Organization for Standardization (ISO). Founded in 1946, ISO is an independent, non-governmental international organisation with a membership of 164 national standards bodies (such as the American National Standards Institute – ANSI, France’s ANFOR and the British Standards Institution – BSI).
In 2010, ISO created a new standard (ISO 29990) targeted specifically towards the design, development and delivery of non-formal education at all levels, including adult training. The standard focused on both providers’ service provision and management systems. However, after recognising the growing complexity and diversity of the global market for learning services, the label was discontinued in 2017. To replace it, ISO created six new standards, each focusing on a particular aspect of quality in non-formal education. Depending on their objectives, contexts and external requirements (such as regulations or contractual arrangements), providers of non-formal training can now apply for a combination of these standards:
ISO 21001 “Management system for educational organisations”
This standard specifies management system requirements for a variety of educational organisations (both formal and non-formal). The standard draws on the following principles: (i) focus on learners and other beneficiaries; (ii) visionary leadership; (iii) engagement of people; (iv) process approach; (v) continual improvement; (vi) evidence-based decisions; (vii) relationship management; (viii) social responsibility; (ix) accessibility and equity; (x) ethical conduct; and (xi) data security and protection.
ISO 29991 “Language learning services outside formal education – Requirements”
This standard is aimed at organisations that provide language courses, solely or alongside other courses, outside formal education. It sets minimum requirements in terms of teaching staff, learning materials and environment, assessments, and advertising, among others. It will soon be replaced by the new – currently under development – standard ISO/DIS 29991.
ISO 29992 “Assessment of outcomes of learning services – Guidance”
This standard provides a framework for the development, implementation and use of results from assessments of learning outcomes. It provides guidance on the selection, validation, planning, administration and use of assessments in a range of learning outcomes.
ISO 29993 “Learning services outside formal education – Service requirements”
This standard specifies requirements for learning services outside formal education, including all types of lifelong learning (e.g. vocational training and in-company training, either outsourced or in-house). In particular, it aims at aligning various elements of learning services, including advertising, information provided to learners, needs analysis, design, assessment and evaluation, for the purpose of improving the effectiveness, efficiency and transparency of learning services.
ISO 29994 “Learning services outside formal education – Additional requirements for distance learning”
This standard is currently under development and not available yet.
ISO 29995 “Learning services outside formal education – Terminology”
This standard is currently under development and not available yet.
As all the standards developed by ISO are covered by copyright and patent laws, public access to their content in terms of minimum quality requirements is not available. For this reason, it is not possible to gauge the value of these standards and, therefore, are not considered further.
Although each quality assurance system has its own functioning, some practices are common to most certification processes. Indeed, four steps are regularly at the basis of certifications and quality label systems (Broek and Buiskool, 2013[2]). Firstly, providers of adult training must prepare their application dossier, filling up various forms and submitting a request to be quality assured. Frequently through self-evaluation reports, providers must assure that they comply with the standards requested by the certification organism. In a second step, the responsible body – be it a public, semi-public or private agency – carries out an external evaluation to assess the fulfilment of the required quality standards. This assessment can involve both on-site visits and inspections, expert consultations, and a validation of providers’ self-reports. The third step is the approval of the application by the responsible body and the provision of the quality seal. Note that in many cases, the approval process by the certification organism is not merely “approved” or “not approved”, but it involves scales of merit or conditional approval decisions.7 The fourth and last common step in certification processes is the monitoring and follow-up of the approved quality seal. This may involve annual reports, on-site visits and – typically – the renewal of the label, if the certification was valid only for a limited time period.
For instance, in order to obtain the Ö-Cert label in Austria, candidate providers follow the steps outlined in Figure 2.1. First, the candidate institution needs to provide evidence of its identification as a provider of adult education, thereby fulfilling basic requirements concerning its organisation, offer and principles of ethics. In a second phase, the application is reviewed by the Ö-Cert Office.8 In case there are documents missing or anything else is unclear, providers need to provide clarifications. In the third step, an accreditation group of experts9 control using a checklist that the application of the provider is valid. The most important selection criterion here is the existence of external audits. This step therefore includes confirming that providers have one of the 11 valid quality management systems or quality assurance procedures included in the Ö-Cert list. If the request of accreditation is accepted, the provider is registered as one of the quality providers of adult education in Austria and receives the Ö-Cert label (after payment of an administration fee of EUR 100). As the Ö-Cert certification is only an umbrella label, its duration is the same as the duration of the quality management system that the providers have obtained (with a 6‑month tolerance limit), although the validity period is not made available to the public.
In a similar vein, providers of adult training applying for the eduQua label in Switzerland have to compile an application dossier following specific guidelines, where they need to prove their fulfilment of the minimum standards. The process differs from the Austrian initiative because there is an on-site audit, which has to be conducted before the evaluation of the dossier. The eduQua label is only awarded after the provider successfully passes the on-site audit and the dossier evaluation.
The eduQua label is also peculiar in its organisation, as pictured in Figure 2.2. In fact, the certification procedure is not performed by the eduQua office but by seven external certification bodies. These bodies are the ones evaluating in practice providers’ fulfilment of the eduQua requirements, undertaking on-site audits, complaint handling, and helping providers meet the quality requirements. In order to become certification bodies of the eduQua label, organisations must be accredited by the Swiss Accreditation Service (SAS). Not only SAS assesses the competence of the certification bodies and their personnel for the correct performance of certifications (i.e. compliance with the rules of procedure), but it also ensures that the procedures of certification bodies remain satisfactory over time. Indeed, collusion between certification agencies and providers is a non-negligible risk, especially in a context where the former cannot compete on prices (certification fees are fixed by the eduQua Advisory Group). Relying on external bodies for the certification procedure allows eduQua to have only a small team of permanent people directly employed in their office, whose role is really to coordinate and ensure a smooth running of the organisational machine. This cuts eduQua office’s running costs significantly, including also marketing expenses to publicise the eduQua label, since these expenses are paid directly by the certification bodies. As a result, the eduQua office manages to run by charging the certification bodies only roughly 10% of the overall certification fees paid by providers.
Source: Adapted from EduQua (2012[8]) Manual eduQua: 2012: Information regarding the proceeding instruction for certification, https://alice.ch/fileadmin/Dokumente/Qualitaet/eduQua/eduQua_Manual_2012_E.pdf.
While, as shown above, most certification processes follow similar steps, identifying common quality standards used in the various certification systems is no easy task. In fact, detailed information about the specific criteria are often not publicly available, and – even when it is – quality standards vary considerably across contexts and quality assurance systems. Some focus more on accountability of the provider in terms of efficiency of how the public funds are spent, others concentrate on the quality of the didactics, and others instead prefer to focus on the quality of the learning outcomes.
Overall, four broad categories of quality criteria can be identified in certification processes (Broek and Buiskool, 2013[2]):
First, quality standards on the organisational structure and management of the providers are almost ubiquitous in all certification processes across Europe. Typically, this set of criteria aims at ensuring that the provider has a well-defined and appropriate mission, its organisation is structured properly with solid management practices, the physical infrastructure of the provider is suitable for adult learning, and its finances are administered efficiently.
Second, quality standards on the teaching staff of the providers are also frequently encompassed in most certification processes. Such type of standards includes setting minimum qualifications or competence levels and offering further training for the adult trainers themselves.
Third, quality of didactics and the learning process, although not all certification processes stress this aspect equally. The goal of these specific standards is to make sure the educational offer of the provider fits with the needs of adult learners. This third category of quality criteria includes for example guidance and counselling for learners, complaint procedures, assessments of the quality of exams and evaluations, education and training methods, etc.
Fourth, quality of training outcomes, which is included in most certification processes to evaluate providers. Relevant criteria include follow-up evaluations of training programmes, feedback from students, labour market performance of past learners, etc.
The case of the eduQua label in Switzerland illustrates well the sort of quality standards that certification processes use in practice to evaluate adult education providers. To measure the quality of a provider, eduQua uses 22 well-defined standards grouped in six criteria, which are all listed in a publicly available manual (EduQua, 2012[8]). For each of the 22 standards, the manual precisely defines:
The objectives of the standard and why it has been selected;
The requirements to respect;
The indicators that can be used to evaluate the standard;
The documents that the provider needs to send to the eduQua agency during the certification process; and
The documents that will be needed when the certification agency will perform an on-site audit at the provider’s premises.
The list of 6 criteria and 22 standards is presented in Table 2.1, while Box 2.2 shows an example of how detailed is the discussion of criteria and standards in the eduQua manual.
It is interesting to note how the eduQua quality criteria have already been subject to three revisions since the creation of the label, with a new revision planned for 2021, in order to improve and meet the fast-changing nature of the adult education landscape. Among the latest changes, one that seems particularly important is the move away from more standardised, rigid teaching/learning processes to the facilitation of individual, independent learning pathways – in line with the latest recommendations from the European Union.
Source: EduQua (2012[8]), Manual eduQua: 2012: Information regarding the proceeding instruction for certification, https://alice.ch/fileadmin/Dokumente/Qualitaet/eduQua/eduQua_Manual_2012_E.pdf.
The level of granularity that the eduQua manual provides for each standard can be illustrated taking one particular standard as an example. For instance, standard 21 “Customer satisfaction” has been chosen in order to provide comprehensive and good quality customer services. Participant satisfaction should not be limited to aspects of training, but also take into account other important features of the institution, such as its infrastructure and its websites. The requirements for this standard include, among others, the presence of regular customer surveys and an efficient complaint service. The indicators used to evaluate this standard refer to the correct implementation of improvement measures following the evaluation of customers’ satisfaction.
The documents that the provider needs to include in its application package should contain information on the measurement, evaluation and development of customer satisfaction over the previous three years, also specifying the exact instruments and methods used. When the eduQua agency carries out the on-site audit, it checks not only the result of the assessment of customer satisfaction, but also the improvement measures implemented. Moreover, the manual also provides additional questions and themes that may be discussed during the on-site visit, such as the main additional benefits for participants (e.g. free parking near the institution, cafeteria, etc.) and how these can be taken into account in the customer satisfaction evaluation.
As shown, the level of details for each criteria used in the eduQua certification process is very high, and it is important to acknowledge the effort put by the decision makers behind the eduQua label in providing such clear and transparent evaluation guidelines. This level of details is particularly important in the Swiss context to ensure that the different certification agencies use the same standards to evaluate providers.
Source: EduQua (2012[8]), Manual eduQua: 2012: Information regarding the proceeding instruction for certification, https://alice.ch/fileadmin/Dokumente/Qualitaet/eduQua/eduQua_Manual_2012_E.pdf.
The basic requirements for acceptance into the quality framework for adult education Ö-Cert in Austria are slightly less detailed yet still clearly defined. The provider has to fulfil five categories of basic requirements: general basic requirements, basic requirements concerning the organisation of the provider, basic requirements concerning the offers of the provider, basic requirements concerning principles of ethics and democracy, and basic requirements with regard to quality (Table 2.2). Providers need to submit a mission statement, an organisation chart, terms of business, proofs from a person with pedagogical knowledge, the course program and a valid quality certificate.
1. The 11 valid external certificates of quality are: ISO 9001: 2008, ISO 29990: 2010, EFQM, LQW (Learner-Oriented Quality Certification for Further Education Organizations/Germany), QVB (Quality development in the array of educational institutions/Germany), EduQua (the Swiss quality label), UZB (environment-label/Ministry of Agriculture), four quality management systems of Austrian federal states: Salzburg, Vienna, Upper Austria, Lower Austria. The main common feature of these different certifications is that they all have an external audit.
Source: Ö-Cert (2019[9]), Ö-CERT [AT-Cert]: An overall framework of quality for Adult Education in Austria, https://oe-cert.at/media/OE-Cert_abstract.pdf.
In a similar vein, the recent French Qualiopi certification is based on the new national quality reference system (“Référentiel national de certification qualité des actions concourant au développement des compétences” – RNCQ), which is also very detailed. The RNCQ is organised around seven criteria linked to 22 indicators which apply to all providers (common core), to which 10 other indicators are added specifically for apprenticeship and training leading to a certification (Table 2.3).10 Box 2.3 presents another example of quality criteria used for a quality certification of career guidance services rather than for adult learning.
Source: French Ministry of Labour (2020[10]), Référetiel National Qualité mentionné à l’article L. 6316‑3 du Code du Travail, https://travail-emploi.gouv.fr/IMG/pdf/guide_referentiel_qualite_28-02.pdf.
Launched in 2002 by the British Department for Education, the Matrix Standard is a quality certification assigned to organisations that deliver information, advice and/or guidance (IAG) services, either as their sole activity or as part of their wider offering. It aims at supporting individuals in their choice of career, learning, work and life goals by ensuring that IAG providers meet a certain level of competency. It helps providers improve their services by benchmarking against best practices and it offers certification to those that meet the full standard. Organisations that can apply to the certificate include training providers, universities, voluntary and community organisations, and private businesses. While the Standard is typically voluntary, it becomes a prerequisite to access public funding in particular cases.
The Matrix Standard is outcome-based, which means it does not focus only on processes used to support IAG delivery but also looks at results achieved. Overall, an organisation needs to prove that: (i) they know and clearly define what they offer to their learners; (ii) they provide accurate, impartial, up-to-date information; (iii) they allow learners to make informed decisions; and (iv) they continuously evaluate and improve the training and information provided. In particular, the 27 quality criteria that Further Education Establishments need to meet are grouped into the following headings:1
Leadership and management: the services provided have precise objectives and clear leadership; the organisation complies with existing legislation, operates with integrity and cooperates with other bodies; clients’ outcomes are at the centre of the services, and their outcomes measure the success of the service.
Resources: the organisation uses its resources effectively; clients are provided with accurate information; staff has the appropriate qualification and are supported in continuous professional development.
Service delivery: the services provided are impartial and effective to meet predetermined objectives; clients are responsible for making their own decisions and, when exploring options, are supported with appropriate resources, including and referral to other appropriate organisations.
Continuous quality improvement: the organisation evaluates its services against its objectives and identifies improvements; both customers’ satisfaction and staff performance are also evaluated; effective use is made of technology to improve the service; the quality assurance process is continuous and dynamic.
1. The complete manual with the quality criteria, instructions for assessments and example are available at: https://matrixstandard.com/media/1058/the-matrix-standard-guidance-for-fe-establishments-20190107.pdf.
Yet, not all certification initiatives need to have such level of details and granularity in terms of their quality requirements. Less complex quality assurance mechanisms may also have some advantages. For instance, there is evidence that consumers often react more to information that are easier to understand than to more sophisticated – but perhaps more objective – quality systems (Dafny and Dranove, 2008[11]). An example of more straightforward quality labels is the label de qualité created in 2000 for the non-formal adult education sector in Luxembourg. The aim of the label is twofold: improving providers’ quality, as well as supporting the monitoring of the otherwise difficult-to-regulate non-formal sector, allowing the government to gather data and information. The requirements that providers of adult training need to meet in order to obtain the label de qualité of Luxembourg are only ten, and they refer mostly to organisational aspects of the training (e.g. minimum number of participants in each course and special fees for disadvantages groups) or staff qualifications. Similarly, the Eduform label in France is based on a high level of requirements for 13 commitments, as well as for the organisation and management of the targeted structures (Box 2.4).
The requirements of the label de qualité in Luxembourg
1. A minimum of 15 learners (exceptions are possible for certain courses);
2. Guaranteeing general access to the courses;
3. Availability of a special enrolment rate (EUR 5 to 10) to disadvantaged target groups;
4. Publishing the courses including information on learning outcomes;
5. Advising the learners to find the right offer;
6. Providing information about accessibility for persons with special needs;
7. Applying a pedagogical approach based on the needs and the situation of adults;
8. Teachers must be accredited by the Minister (pedagogical and content-related competences achieved either through initial education, continuous education or professional experience);
9. Delivering a participation certificate to learners who attend 70% of the course;
10. Delivering on demand an individual certificate including the description of skills and knowledge obtained.
The commitments of the Eduform label in France
1. Quick and guided access to information on the services offered;
2. Quality of the reception;
3. Personalised advice on the services and their financing possibilities;
4. Proposal of a wide range of services;
5. Help in building a tailor-made response;
6. Contractualisation with the beneficiary on the objectives, the contents and the modalities of the services;
7. Support, monitoring, evaluation and readjustment of the beneficiary’s journey throughout the service by a dedicated person;
8. Adaptation to each service and for each beneficiary of the premises and the pedagogical means, methods, supports, tools and materials of the service;
9. Assessment and recognition of prior learning and certification;
10. Qualifications and competencies guaranteed and developed throughout life;
11. Taking into account the satisfaction of customers and beneficiaries;
12. Continuous improvement of services and trainings;
13. Taking into account stakeholder expectations of corporate social responsibility and sustainable development.
References
[3] Banerjee, A. and B. Solomon (2003), “Eco-labeling for energy efficiency and sustainability: A meta-evaluation of US programs”, Energy Policy, Vol. 31/2, pp. 109-123, http://dx.doi.org/10.1016/S0301-4215(02)00012-5.
[2] Broek, S. and B. Buiskool (2013), Developing the adult learning sector: Quality in the Adult Learning Sector.
[4] Cashore, B., G. Auld and D. Newsom (2003), “Forest certification (eco-labeling) programs and their policy-making authority: Explaining divergence among North American and European case studies”, Forest Policy and Economics, Vol. 5/3, pp. 225-247, http://dx.doi.org/10.1016/S1389-9341(02)00060-6.
[11] Dafny, L. and D. Dranove (2008), “Do report cards tell consumers anything they don’t already know? The case of Medicare HMOs”, RAND Journal of Economics, Vol. 39/3, pp. 790-821, http://dx.doi.org/10.1111/j.1756-2171.2008.00039.x.
[8] EduQua (2012), Manual eduQua: 2012: Information regarding the proceeding instruction for certification, https://alice.ch/fileadmin/Dokumente/Qualitaet/eduQua/eduQua_Manual_2012_E.pdf.
[1] France Strategie (2013), Le marché de la formation professionnelle continue à l’épreuve de l’enjeu de la qualité.
[10] French Ministry of Labour (2020), Référetiel National Qualité mentionné à l’article L. 6316-3 du Code du Travail, https://travail-emploi.gouv.fr/IMG/pdf/guide_referentiel_qualite_28-02.pdf.
[5] Gruber, A., S. Brünner and E. Huss (2009), Perspektiven der Erwachsenenbildung im Rahmen des lebenslangen Lernens in der Steiermark (PERLS).
[9] Ö-Cert (2019), Ö-CERT [AT-Cert] - an overall framework of quality for Adult Education in Austria, https://oe-cert.at/media/OE-Cert_abstract.pdf.
[7] OECD (2019), Getting Skills Right: Future-Ready Adult Learning Systems, Getting Skills Right, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264311756-en.
[6] OECD (2019), OECD Skills Strategy Flanders: Assessment and Recommendations, OECD Skills Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264309791-en.
← 1. ANFOR is the French Standardisation Association (Association Française de Normalisation), which is in charge of developing international standardisation activities, information provision and certification in France. Its standard BP X50‑762 proposes criteria of quality for the services offered by providers of continuing professional training. It specifies, in the organisational provisions, their conditions of implementation and mentions how the management intends to define, promote, monitor and improve its policy in terms of quality of service.
← 2. http://www.cnefop.gouv.fr/qualite/liste-des-certifications-et-labels-generalistes-du-cnefop.html.
← 3. Without this new certification, organisations can only work as training provider for companies managing their own skills development plans.
← 4. While initially set to 1 January 2021, the date by which all providers need to have the new certification has been postponed by a year, due to the COVID‑19 crisis.
← 5. The training organisation can at this stage change certification body.
← 6. The decree text can be found here: https://data-onderwijs.vlaanderen.be/edulex/document.aspx?docid=15324.
← 7. For example, in Switzerland the eduQua label can be granted unconditionally or conditionally under certain conditions that are clearly stipulated. Similarly, in France the committee in charge of assigning the Eduform label has four options: a) grant the label for 3 years; b) deny the label; c) demand additional information; and d) demand another audit.
← 8. Only four people work in the Ö-Cert Office. Their role is to make the first check of the applications and be responsible for the central organisation of the label.
← 9. The accreditation group is a group of five experts who are responsible for the accreditation of providers; they assess the proofs and meet approximately 5‑7 times a year.
← 10. The full manual (in French) is available here: https://travail-emploi.gouv.fr/IMG/pdf/guide_referentiel_qualite_28-02.pdf.