Mapping Quality Assurance Indicators for Non-formal Adult Learning

3. Planning

Abstract

In the first phase of the EQAVET quality cycle, the planning phase, institutions set up clear, appropriate and measurable objectives, which support the development and quality improvement of training providers and the training system. This chapter looks at the quality areas, criteria and evidence that training providers can submit to prove compliance for all quality areas related to EQAVET’s indicative descriptors included in the planning phase.

The first phase in the EQAVET quality cycle is the planning phase. In this phase, institutions set up clear, appropriate and measurable objectives to support the development, evaluation and quality improvement of the VET system and of training providers.

To guide institutions in implementing the planning phase, EQAVET proposes indicative descriptors, which outline the success factors needed to design high quality training. EU member countries developing new quality frameworks for VET can choose, from these descriptors, the most suitable for their national context. To align existing quality assurance systems to the EQAVET framework, countries with QA systems already in place can add, update or drop some of their indicators. Countries may also decide to include quality indicators in their frameworks that are not considered by EQAVET, for example on the learning infrastructure.

This chapter presents the quality criteria that the QA systems reviewed for this report use in the planning phase. These criteria can be broadly grouped into three quality areas: leadership and strategy, training programme and learning infrastructure and environment. The extent to which each system analysed, including EQAVET, covers these areas is presented in Table 3.1. Darker shades indicate that a higher number of indicators is used to measure quality in that area by a particular QA system, reflecting the importance attached to that area.

As presented in Table 3.1, while the EQAVET framework covers two of the three planning quality areas, most of the systems analysed in the report (7 out of 11) cover all three areas. The leadership and strategy area, which includes general organisation management criteria, and the training programme area, covering criteria like identifying training needs, are covered by almost all systems analysed. However, the learning infrastructure and environment quality area is not covered by 4 of the systems reviewed, including EQAVET.

Table 3.1. Quality areas covered by the different QA systems

Quality area	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Leadership and strategy					(QMS)
Training programme
Learning infrastructure and environment					(QMS)

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Darker shades represent a higher number of indicators related to that quality area for a given framework.

Source: Authors’ elaboration.

The following sections describe each one of these quality areas, including a presentation of the relevant EQAVET indicative descriptors at provider level, an overview of the criteria included in each quality area and the documental evidence that providers can submit to prove compliance.

Leadership and management

Most EQAVET indicative descriptors for the planning phase concern leadership and management (as presented below), which highlights the relevance of this quality area. The leadership and management quality area is the basis of planning, setting up provider and system objectives, involving stakeholders in the system and ensuring that minimum policies have been set up.

EQAVET indicative descriptors at provider level:

• European, national and regional VET policy goals/objectives are reflected in the local targets set by the VET providers.

• Responsibilities in quality management and development have been explicitly allocated.

• VET providers have an explicit and transparent quality assurance system in place.

• Providers plan co‑operative initiatives with relevant stakeholders.

• Measures are designed to ensure compliance with data protection rules.

• There is an early involvement of staff in planning, including with regard to quality development.

Given the importance of this quality area, a significant share of the QA systems reviewed cover multiple criteria linked to the leadership and management quality area (see Table 3.2). These often overlap with the indicative descriptors outlined in EQAVET, which cover five of the seven quality criteria attributed to this area. Indeed, of the 11 QA systems reviewed, 9 include criteria related to leadership and management, with most criteria relating to general organisation management, which is not specific to the CVET sector. Based on the use by the systems reviewed, the most relevant criteria in this area are related to defining objectives, setting up a quality management system and collecting reliable information and data. On the other end, only one system, EduQua, includes indicators on planning a risk and opportunities management strategy.

Table 3.2. Leadership and management quality criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Objectives	x	x	x		x (QMS)			x	x		x
Quality management system	x	x	x		x	x		x	x
Organisational structure		x	x		x
Management of risks and opportunities		x
Communication and information	x	x	x			x					x
Reliable information and data	x	x		x		x			x		x
Stakeholder engagement	x		x			x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Authors’ elaboration.

Each one of the quality criteria included in the leadership and management area are described in the following paragraphs, including how these criteria are implemented and the goal of each criterion.

Quality frameworks typically require training providers to formulate their objectives or a mission statement, including a description of the context in which they operate, such as their positioning in the training market, potential target groups and labour market needs (e.g. EduQua and DGERT). This step aims at aligning the providers’ activities with the needs and constraints of the population they will serve as well as the environment in which they operate. Often, this criterion is complemented by the requirement that training providers publicise this information through a communication strategy, ensuring transparency and the informed involvement of stakeholders, as in the case of QQI.

The use of a quality management system is generally required by quality frameworks (e.g. Ö-Cert, the NRTO quality label), as it ensures the continuous quality improvement of the provider as well as the development of an internal quality culture. These systems set framework criteria, requiring the arrangement of quality assurance processes and objectives and informing all stakeholders of their existence; determine the human and financial resources needed for quality assurance; and set the responsibilities and involvement of staff. In addition, several quality assurance frameworks require a systematic analysis of the quality management system findings in the review phase (as discussed in Chapter 6).

Training providers must also define their organisational structures in a few of the QA frameworks analysed (e.g, Cert NÖ, Ö-Cert, OQEA), and match them with their objectives and services. This criterion is frequently linked to establishing guidelines for the decision-making processes in the organisation, determining the responsibilities and operation of management bodies, providing for ongoing review of the organisational structures and promoting communication between different organisational areas, as in EduQua. In the case of the NRTO quality label, training providers are also encouraged to ensure continuity of trainees’ learning processes in case of organisational or institutional changes.

Planning a risks and opportunities management strategy may also be included as a QA criterion, as in EduQua. Training providers’ management is expected to consider personnel, organisational, technological and financial risks and opportunities, evaluate and monitor their probability and potential impact, take preventive measures if possible and implement damage limitation measures in the case of risk events.

Several of the quality systems analysed include indicators on defining communication and information flows within the organisation. These systems, such as QQI, expect providers to ensure that staff is kept informed about anything concerning them or the programmes they are involved in. This criterion could also include, as in EQAVET, involving staff in the planning phase, to ensure that they have the information they need to carry out their functions.

Another quality criterion often included in QA frameworks is the collection of reliable information and data for informed decision-making (e.g. the NRTO quality label, EQM). At the planning stage, this refers to setting up data collection and processing so that staff and management can use this information for self-monitoring and planning purposes. The information collected through these structures is generally used during the evaluation phase, discussed in Chapter 5. Some examples of data collected include trainees’ assessment results, subject choices, or programme completion. In addition, some systems specify data protection criteria, i.e. that the training provider manages the creation and storage of the documented information or that the provider ensures data and privacy protection and protects the information from loss and misuse (e.g. the NRTO quality label and QQI).

Finally, while the EQAVET framework and the adult learning literature (OECD, 2019[8]) attribute a lot of importance to multi stakeholder engagement, this seems to be less relevant in the other QA systems analysed. QQI includes indicators on whether the training provider engages with the broader education and training community, for example. However, other systems, such as Qualiopi and the NRTO quality label, include stakeholder engagement but at the end of the quality cycle, in the review phase (discussed in Chapter 6). Engaging stakeholders in the planning phase, as suggested by the EQAVET framework, could allow training providers, for example, to plan training initiatives in co‑operation with stakeholders.

The fulfilment of the leadership and management criteria included in the relevant quality assurance framework is reviewed through documentation provided by the training provider. EduQua, for example, lists a range of mandatory documents per quality criterion, as well as several optional ones. Table 3.3 shows the different documents that QA systems require providers to hand in to prove compliance with each criterion.

Table 3.3. Evidence used for leadership and management criteria

Quality criteria	Evidence
Overall management, objectives	Strategic goals of the institution Mission statement Priority objectives and planned activities for a defined period of time Compilation of management and control instruments Balanced scorecard Documentation of the relevant inputs and results from internal and external audits as well as the specified parameters of the last year Self-evaluation with respect to the quality framework criteria Stakeholder and/or environment analysis
Quality management system	Quality concept or presentation of the quality management system Documentation of an implemented and evaluated quality project Self-evaluation
Organisational structure	Overview of the organisational structure (organisation chart) Function or role descriptions Guidelines on decision-making processes General Terms and Conditions of Business
Management of risks and opportunities	List and assessment of risks and opportunities Examples of preventive or corrective measures Audit report or income statement/balance sheet
Communication and information	General Terms and Conditions of Business Overview of the organisational structure (organisation chart)
Reliable information and data	Guidelines for the documentation and storage of information in the training institution Regulations on data protection and compliance with personal rights Guidelines on the secure handling of data in the institution
Stakeholder engagement	Mission statement Self-evaluation Interviews

Source: Authors’ elaboration.

Training programme design

Several EQAVET indicative descriptors in the planning phase aim at ensuring the quality of the training programmes offered by training providers. These descriptors pay particular attention to the design and description of the training programmes and to the alignment of training programmes with local, individual and labour market needs.

EQAVET indicative descriptors at provider level:

• Explicit goals/objectives and targets are set and monitored, and programmes are designed to meet them

• Ongoing consultation with social partners and all other relevant stakeholders takes place to identify specific local/ individual needs

• The relevant stakeholders participate in the process of analysing local needs.

Since the training programme is at the core of the services offered by training providers, almost all QA systems covered in this report define criteria to strengthen the quality of training programme design (Table 3.4). Some QA systems offer detailed guidance on the design of the training programmes, like OQEA or QQI, while others give more flexibility to training providers, as EQM.

As shown in Table 3.4, most quality frameworks include indicators related to the identification of training needs and the development of the training programme, however, only four of the systems analysed include references to training programme supporting processes (the NRTO quality label, Ö-Cert, DGERT and Label de Qualité).

Table 3.4. Training programme quality criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21002/ 9001
Identify training needs	x	x	x		x		x		x	x	x
Develop the training programme	x	x	x			x	x	x	x	x
Supporting processes				x	x(QMS)			x		x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Authors’ elaboration.

With regard to the first quality area, in line with the EQAVET indicative descriptors, most quality frameworks require providers to first identify the training needs and requirements in the labour market and of the programme’s target population (e.g. Qualiopi, EQM). This is an important step to define the programme training area, the competences and knowledge that will be developed and to design an appropriate learning environment. This identification process can be done, for example, by involving learners and other stakeholders in the design of the programmes, by consulting external experts and social partners on labour market needs and by aligning training programmes to qualifications included in the National Qualification Framework. This criterion is an important prerequisite for developing a learner-centred training programme, as it takes into account the needs of the programme’s target population.

The second set of criteria relate to developing the training programme. At this stage, providers are required to ensure that the programme area and teaching methods are based on the identified needs (e.g. EduQua and OQEA). In addition, training providers must take into account potential developments that could affect training needs (such as foreseen changes in the local labour market). Particularly, this criterion involves defining the training programme’s learning outcomes, possible training itineraries and considering potential educational methods, programme activities and technical-pedagogical resources, as, for example, in DGERT.

Finally, a range of criteria on processes supporting the programme are often included in quality frameworks. These criteria ensure, for example, that providers define and publish supporting policies for their operation such as, payments and possible refunds and attendance and repetition policies, as DGERT, procedures for handling complaints, as the NRTO quality label, or information on the forms of enrolment they offer (full- or part-time, in person or remote etc.) and the times and places where the courses will take place. The definition and publication of these supporting policies increases transparency of training provider operations.

Table 3.5 shows the documentation that QA systems require providers to submit in order to prove that they are complying with the quality standards for each criterion in this quality area.

Table 3.5. Evidence used for training programme criteria

Quality criteria	Evidence
Identify training needs and requirements	Procedures or instruments for determining the labour market skill needs and the needs of the target group Written concept of the training programme
Developing the learning programme	Training programme summary or detailed description of its content, form (course, seminar, lecture), learning objectives, number of teaching units Description of the form, content, scope and frequency of information dissemination to existing and prospective trainees Specific didactic guidelines
Processes around the programme	Relevant policies (attendance, repetition, payments, enrolment) and procedures (handling of complaints)

Source: Author’s elaboration.

Learning infrastructure and environment

The last quality area included in the planning phase refers to learning infrastructure and environment. The EQAVET indicative descriptors do not explicitly refer to this quality area. However, this area is addressed by most quality frameworks and seems to play a role in the adult learning experience. As discussed in Box 3.1, the andragogy literature, the literature on creating effective adult learning, has referred to the need of offering “physical conditions that are comfortable (as to seating, and smoking, temperature, ventilation, lighting, decoration) and conducive to interaction (preferably, no person sitting behind another person)” (Knowles, Holton III and Swanson, 1973[9]) to facilitate adult learning. In addition, other reasons such as safety, health or accessibility can imply the need to have a fire escape staircase, access to windows in all classrooms or a stair-free access.

Box 3.1. Andragogy: The art and science of helping adults learn

Teaching adults is quite different from teaching children. For example, adults must experience a need to learn and feel that what they learn is applicable, while children are open to learning what society determines they should learn and understand that most of the learning content will only be useful later in life (Knowles, 1970[10]). Given these differences, a strand of the education literature, called andragogy, has focused on understanding the adult learner and developing strategies to improve the effectiveness of teaching adults. Andragogy studies “the art and science of helping adults learn” (Knowles, Holton III and Swanson, 1973[9]) and is based on six assumptions, which describe how the adult’s learning experience differs from the child’s learning experience.

These assumptions imply that, for adults to be ready to learn, they must feel the need to know something. In addition, during the learning process, adults should also be treated as capable of making their own decisions, adults’ experiences should be used in the learning process and the content of the programme must be perceived as useful and applicable and be aligned with adults’ goals and life circumstances.

As a result of these assumptions, adult learning should follow some principles, discussed in Knowles, Holton III and Swanson (1973[9]). These refer mostly to the teacher-learner interaction, but also include guidance on adults’ participation in the programme design, classroom dynamics, student evaluation and on how the classroom should be set up to be conducive to learning.

The extent to which these principles can be applied in an actual programme depends on some factors, such as the structure of the content to be taught, the level of student maturity, the objectives of the students, the class size and how often the class meets and for how long (Birzer, 2004[11]).

The andragogical teaching principles have been applied to a variety of settings, such as programmes on criminal justice (Birzer, 2004[11]), medical training (Bedi, 2004[12]), police training (Birzer, 2003[13]), and management (Forrest and Peterson, 2006[14])

For these reasons, as shown in Table 3.6, many of the QA systems analysed include indicators to ensure that the learning infrastructure is of high quality (e.g. EduQua or QQI). This infrastructure can refer to the quality of the physical learning facilities, to offering a conducive learning environment or also to the virtual infrastructure for online learning (e.g. the NRTO quality label), depending on the system. In addition, quality frameworks often contain requirements on the technical equipment of the training provider.

Table 3.6. Learning infrastructure and environment criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21002/ 9001
Infrastructure		x	x		x (QMS)	x	x		x	x
Equipment		x	x		x (QMS)	x	x	x	x	x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Author’s elaboration.

Following the andragogy literature, most quality frameworks analysed require study premises to be suitable for the educational programmes that are offered, considering pedagogical needs and adapted to the characteristics of adult learners. Some frameworks are more specific and require that the study premises allow for the use of active work methods, which are more suitable for adults, that classrooms are of an appropriate size for the number of trainees (e.g. Cert NÖ, DGERT, Label de Qualité), that they have adequate lighting, heating, maintenance and cleanliness and that they are accessible and adapted for adults with different levels of mobility or special needs (e.g. OQEA). Training providers’ infrastructure should also comply with safety, health, ecology and hygiene legal requirements (e.g. EduQua). Some quality frameworks also require the existence of dedicated spaces for staff or trainees to work in, such as staff common rooms or spaces for individual work.

Depending on the training programme, providers are also required to supply technical and non-technical equipment (e.g. DGERT). This may include video projectors, whiteboards, computers, language rooms, technical platforms, remote connections, a documentation space or a teaching platform. The availability of information and communication technology is also required by some QA systems, as EduQua. Additionally, as online learning becomes more widespread, some QA frameworks specifically emphasise the need for providers to accommodate this learning option, as the NRTO quality label.1

Compliance with criteria referring to learning infrastructure and environment can be proven through the supporting documents outlined in Table 3.7.

Table 3.7. Evidence used for learning infrastructure and environment criteria

Quality criteria	Evidence
Infrastructure	Guidelines on teaching, learning spaces and learning infrastructure Training concepts Safety policies Description of the pedagogical requirements of the course rooms for the implementation of the training Location of the premises (exact address of all premises) Description of the size and equipment of the course rooms (technical equipment, seating, etc.)
Equipment	Specific guidelines for the selection and use of technology-based learning environments Training concepts Description of the size and equipment of the course rooms (technical equipment, seating, etc.)

Source: Author’s elaboration.

Note

← 1.  Chapter 4 provides more details on the requirements for access to online learning and guidance platforms and electronic materials.