Mapping Quality Assurance Indicators for Non-formal Adult Learning

4. Implementation

Abstract

In the implementation phase, training providers set up procedures to ensure the achievement of the objectives defined in the planning phase. Quality areas related to this phase include criteria on the training provider’s staff; on implementing the training programme designed in the planning phase; and on offering guidance and information to trainees and other stakeholders. Based on EQAVET’s indicative descriptors for the implementation phase, this chapter analyses the quality areas, criteria and evidence that training providers can submit to prove compliance that the quality assurance systems analysed for this report most commonly use.

The second phase in the EQAVET quality assurance cycle is the implementation phase. During this phase, procedures are put in place to ensure the achievement of the goals and objectives that were established during the planning phase. Quality assurance systems ensure that the design of these procedures leads to high quality training provision.

As for the planning phase, EQAVET proposes indicative descriptors for the implementation phase from which countries can choose. These can be broadly grouped into three quality areas: training provider staff; training programme; and guidance and information. Table 4.1. shows the extent to which the QA systems reviewed for this report cover each one of these quality areas.

With the exception of the two ISO labels, 21001 and 9001, all the systems analysed in this report include criteria on training provider staff and on guidance, information and outreach. Generally, quality assurance systems cover the area of training provider staff in more depth than the area of guidance and information, as it is part of the core services of the training provider while guidance is often organised by external guidance providers. While the quality area referring to the training programme is relatively important in the systems that include it, five out of the 11 systems do not include criteria in this area.

Table 4.1. Quality areas covered by the different QA systems

Quality area	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Training provider’s staff					(QMS)
Training programme
Guidance and information

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Darker shades represent a higher number of indicators related to that quality area for a given framework.

Source: Author’s elaboration.

Training provider staff

Given the pivotal role of trainers in the learning process, the EQAVET framework includes several indicative descriptors related to staff in the implementation phase. These descriptors refer to staff recruitment and distribution of work, collaboration between teachers and staff development.

EQAVET indicative descriptors at provider level:

• Resources are appropriately internally aligned/assigned with a view to achieving the targets set in the implementation plans

• Relevant and inclusive partnerships, including those between teachers and trainers, are explicitly supported to implement the actions planned

• The strategic plan for staff competence development specifies the need for training for teachers and trainers

• Staff undertake regular training and develop co‑operation with relevant external stakeholders to support capacity building and quality improvement, and to enhance performance.

Similarly, all the systems analysed for this report contain indicators referring to the staff working for training providers (Table 4.2). These indicators relate mostly to two quality criteria that are included in most systems analysed: staff recruitment and staff development.

Table 4.2. Teaching and guidance staff criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21002/ 9001
Recruitment of new staff	x	x	x	x	x	x	x	x		x
Staff skills development	x	x	x	x	x (QMS)	x	x		x		x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by one of the accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Author’s elaboration.

The recruitment of qualified and suitable staff by the training provider is addressed by most of the examined quality frameworks. Some frameworks, such as EQM, simply require providers to employ suitable staff, while leaving the definition of what suitable means to the provider. Others, as OQEA or the NRTO quality label, require the training provider to develop rules (e.g. codes of conduct) and criteria (e.g. minimum experience) for the recruitment and selection of staff, promoting a fair and transparent recruitment process. Stricter QA systems generally include indicators on areas such as requiring providers to have a relevant induction programme for new staff, having staff assigned to different functions, such as managerial, administrative, counselling or supervisory, or imposing minimum requirements for professionals interested in fulfilling specific roles. Along these lines, some quality systems require teaching staff to have an officially recognised certificate as trainers (e.g. SVEB-Zertifikat FA-M1 for EduQua). If training providers are allowed to hire external suppliers to provide some of the training, they must ensure compliance of the external supplier with the applicable selection criteria as well.

Indicators on staff skills development are also included in most of the reviewed quality systems, in some cases already in the planning phase, as in EduQua, in others in the implementation phase, as in EQAVET, and in a third group even at later phases of the quality cycle, as in the evaluation and review phases (e.g. OQEA, where the focus is on updating staff training according to the identified needs). These indicators usually require the provider to offer opportunities for the professional development of teaching staff, to strengthen the link between education, teaching and research, and to encourage innovation in teaching methods and the use of new technologies. Most indicators focus on ensuring that staff has access to career guidance and skills development opportunities based on their career plan and needs. Indicators also often require defining a feedback mechanism for staff members to help them identify their strengths and areas for improvement. Where applicable, such a feedback mechanism can also serve training providers to prove that they comply with the requirement to devote resources to identify staff training needs, if included.

Training providers can prove compliance with the staff recruitment and development requirements using a variety of documents. Examples are shown in Table 4.3.

Table 4.3. Evidence used for staff-related quality criteria

Quality criteria	Evidence
Recruitment of new staff	Staff profile requirements (education and training, professional and pedagogical experience, other competences) Description of the selection process (chronology, procedure, responsibility) List of all internal and external trainers and learning facilitators (scope of activity, qualifications, experience, length of employment)
Staff development	Planning of training events and staff exchange forums Description of the training activities List of attended external and internal, subject-specific and adult education-relevant training by trainers and learning facilitators in the last 12 months

Source: Author’s elaboration.

Implementation of the training programme

An important part of the implementation phase is putting in place the actual training programme. Building on the design of the training programme and its alignment with individual and labour market needs carried out during the planning phase, the next steps focus on selecting trainees, applying different learning methods and harmonising examinations. Most indicative descriptors of EQAVET refer to how the training programme should be implemented, highlighting the required teaching approach.

EQAVET indicative descriptors at provider level:

• Resources are appropriately internally aligned/assigned with a view to achieving the targets set in the implementation plans.

• VET providers’ programmes enable learners to meet the expected learning outcomes and become involved in the learning process.

• VET providers respond to the learning needs of individuals by using a learner – centred approach which enable learners to achieve the expected learning outcomes.

• VET providers promote innovation in teaching and learning methods, in school and in the workplace, supported by the use of digital technologies and online‑learning tools.

• VET providers use valid, accurate and reliable methods to assess individuals’ learning outcomes.

The quality frameworks reviewed for this report echo the indicative descriptors included in EQAVET (Table 4.4). Out of the five criteria included in this quality area, ensuring fair trainee selection processes and assessment procedures at the end of the programme are among the most widely used. While EQAVET already highlights the importance of trainee‑centredness and responding to trainees’ needs at this stage, this is covered by several of the reviewed QA systems in the evaluation and review phases of the quality cycle. Each of the criteria in Table 4.4 are explained in more detail below.

Table 4.4. Training programme criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Evaluate trainees’ existing knowledge			x	x			x
Trainee selection			x		x		x		x	x	x
Use educational & technological innovations	x					x	x		x
E‑learning				x		x
Examinations	x		x	x	x (QMS)	x		x	x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Author’s elaboration.

Several QA systems require providers to assess trainees’ prior knowledge before the start of the training programme in order to develop personalised upskilling or reskilling pathways, as, for example, Qualiopi and the NRTO quality label. These assessments collect information on trainees’ academic qualifications, language proficiency, and work experience and may also include interviews, aptitude tests, and work trials designed to assess the trainee’s knowledge and skills beyond what can be proven on paper. This assessment of prior learning helps ensure that the training programme is tailored to the individual’s needs and abilities and that additional support can be organised where needed, allowing the trainee to make the most of the training experience.

It is also common for quality frameworks to ask training providers to describe the trainee selection process. This involves ensuring that potential trainees are well informed about the training programme contents and requirements, so that they can make informed decisions. At the same time, it also means promoting a good match between learners and training programmes. Guidance for prospective trainees can improve this match. In addition, depending on the programme, it may also be appropriate to establish selection procedures such as placement tests, entrance interviews, or assessments. All of these procedures aim at ensuring that the trainees’ profiles match the selected training programme. Some QA systems also suggest that, once the provider has a better picture of the target groups’ needs, the training programme should be adapted to the target group if needed. Some systems, such as EduQua, also place particular emphasis on transparency and equal treatment during the selection of candidates.

Updating the training content and methods regularly is fundamental to maintaining the relevance of the training offered. Some QA systems acknowledge this important task and require providers to ensure that they are using the latest educational and technological innovations. Educational innovations can include interactive learning tools such as augmented reality, virtual reality, and simulations; the use of collaborative learning methods such as discussion groups and online forums or even the use of gamification and game‑based learning. In fact, it is under this criterion that some systems include indicators to somewhat align training provision to the principles of andragogy (see Box 3.1). Current technological innovations refer, for example, to using mobile learning tools; artificial intelligence and machine learning; big data analytics; virtual and augmented reality; video conferencing and streaming. Some systems also consider innovation in the context of staff development i.e. encouraging staff to bring in new ideas and preparing them for innovations that require new skills (for example EduQua and OQEA).

Some QA systems, such as the NRTO quality label or QQI, include indicators on the use of e‑learning, pushing training providers to take advantage of the many benefits it can offer, particularly for adults. E‑learning can increase participation in training programmes allowing trainees to take courses whenever (in the case of asynchronous courses) and wherever is most convenient for them. Depending on the type of e‑learning, trainees can also work at their own pace and review material at any time, which can be especially beneficial for adult trainees with competing commitments or responsibilities.

Finally, indicators on trainee examinations are often included in quality frameworks. They may refer to the assessment of trainees, focusing on the assessment materials (e.g. tests and tasks to be completed during the exam), assessment processes (e.g. supervision of tests) and trainee coursework (e.g. assignments, practical tests, project work), or to provider’s records of learner assessment. The information collected via these indicators allows determining whether the assessment is fair, consistent and appropriate and whether it is carried out professionally at all times. Trainee feedback on the assessment, both on its design and administration, also provides valuable information to providers about the pertinence of the evaluation process.

To prove compliance with the criteria related to the implementation of the training programme quality area, training providers must provide some documental evidence, outlined in Table 4.5.

Table 4.5. Evidence used for training programme criteria

Quality criteria	Evidence
Evaluate trainees’ existing knowledge	Results of the assessment of prior knowledge Interview guide Assessment of prior learning materials (quiz, exercises, role‑playing, test) Tools for measuring gaps in terms of skills to be acquired or validated Procedures for positioning trainees and/ or access requirements
Trainee selection	Profile of the accepted learners Training programme syllabus and descriptions Examples of advertisements of offered training programmes Documentation of selection procedures used
Use pedagogical and technological innovations	Monitoring of relevant developments and related documents Participation in conferences, symposia, trade fairs, groups for reflection and analysis of practices Training provider’s membership of a professional network (trade union, federation, forums) Subscriptions to professional journals Changes made to the content of the services offered
Examinations and the communication of the learning outcomes	Description of the exams or any other assessment of learning achievements, including assessment criteria and methods for evaluating learning
E‑learning indicators	Verifiable/ monitored progress and/or participation by means of a Learning Management System (LMS)

Source: Author’s elaboration.

Guidance and information

The last quality area linked to the implementation phase relates to offering guidance and information to trainees and potential trainees. Providing career guidance and the information needed to make informed decisions to prospective and current trainees is essential to involve trainees in the learning process and to ensure that the training programme responds to their learning needs. Two EQAVET indicative descriptors at provider level mention the need to enable learners to achieve the expected learning outcomes, stressing the importance of providing support:

EQAVET indicative descriptors at provider level:

• VET providers’ programmes enable learners to meet the expected learning outcomes and become involved in the learning process

• VET providers respond to the learning needs of individuals by using a learner-centred approach which enable learners to achieve the expected learning outcomes.

Given the relevance of this area to enhance the chances that trainees attain the learning outcomes, most of the quality assurance systems analysed include related indicators. As shown in Table 4.6, most of the systems reviewed expect training providers to provide accurate information, and to support and offer guidance to trainees during the training period. While OQEA and the NRTO quality label also include indicators on training providers’ activities in promoting training, this criterion is not common in the systems analysed.

Table 4.6. Guidance and information criteria

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Provision of information		x		x	x (QMS)	x	x	x	x	x
Support and guidance for trainees	x	x	x	x		x	x		x	x
Promotion of training			x	x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-CERT directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Author’s elaboration.

Indicators on the provision of information by providers generally require training providers to make essential information publicly available for all stakeholders, including students, employers, and regulatory bodies. This includes information on training programmes’ access procedures, objectives, duration, learning outcomes, rates and assessment methods as well as accessibility information for people with disabilities and training provider’s contact details. EduQua and QQI require that the information provided is transparent, up to date, understandable and easily accessible. In these systems, information provided by the training provider must include important aspects of the organisation of the provider, such as sponsorship and current co‑operations, area of activity, educational mission and andragogical guiding principles. Publicly available high-quality information is important for learners to choose the most suitable training programme for them and for employers to evaluate the skills of job applicants. Additionally, the availability of this information can promote the correct functioning of the training market, through increased transparency on the offer of providers.

Career guidance also plays a crucial role in the overall success of training programmes. QA systems include indicators on career guidance to ensure that trainees receive the support they need to make informed decisions about their educational and career paths. Offering career guidance to trainees may additionally increase trainee motivation and engagement, resulting in higher retention and completion rates, and ease re‑entry to the labour market. Qualiopi, for example, requires training providers to support trainees before, during and after the actual training programme. Qualiopi and QQI are the only systems that requires the provision of individual guidance to potential learners as well as to those transitioning to the next stage of their careers after completing the programme, being it going back to work or continue developing their knowledge and skills. In addition, providers subject to Qualiopi are required specifically to provide targeted guidance to trainees with disabilities. Other systems require training providers to provide support during the programme to ensure that trainees achieve the learning outcomes. This is the case, for example, of the NRTO quality label, which uses indicators on whether providers offer personalised study support for e‑learning. EduQua follows a similar approach, requiring training providers to accompany trainees when using technological instruments for learning.

Finally, the way that providers promote their training programmes can also be regulated by QA systems. The NRTO quality label, for example, specifically requires providers to respect the advertising code and to refrain from aggressively competitive language. While other systems do not have specific indicators on how training providers should advertise their training offer, their indicators on how information is provided could be applied to this area as well, ensuring that information provided in this context is truthful and updated.

Compliance with each one of these criteria can be checked by the authority in charge of the quality assurance system using training provider documentation, as presented in Table 4.7.

Table 4.7. Evidence used for guidance, information and outreach criteria

Quality criteria	Evidence
Provision of information	Overview of the training offer Published information on training programmes General terms and conditions
Support and guidance for trainees	Counselling guidelines List of structures / resource persons (regularly updated) providing these services Procedure for managing dropouts and systematic reminders Listing telephone reminders to contact trainees Appointment book to follow up with trainees Tools and methods promoting the involvement of the trainee in its training programme (co-constructed documents, shared spaces) Surveys
Promotion of training	Example of advertisements

Source: Author’s elaboration.