Mapping Quality Assurance Indicators for Non-formal Adult Learning

6. Review

Abstract

The last phase of the EQAVET quality cycle is the review phase. This phase ensures that feedback from the evaluation phase is considered, and that continuous improvement is implemented. It guides providers on using the results of the evaluation phase to develop strategies for improvement. Once the review phase is completed, the quality cycle starts again. This chapter discusses the criteria included in the quality assurance systems analysed related to EQAVET’s indicative descriptors for the review phase, as well as the evidence that training providers can submit to prove compliance with these criteria.

The fourth and last phase of the EQAVET quality cycle is the review phase. This phase supports systems and providers in using the results of the evaluation phase to draw conclusions and develop strategies for improvement, ensuring continuous improvement. The objective of the review phase is to design actions to achieve the targeted outcomes and/or new objectives. Ideally all key stakeholders are involved in this phase, strengthening their commitment to the training system or provider’s improvement. Once the review phase is completed, the quality cycle starts again with the planning phase.

EQAVET proposes indicative descriptors to successfully carry out the review phase, from which countries can choose. They are the following:

EQAVET indicative descriptors at provider level:

• Learners’ feedback is gathered on their individual learning experience and on the learning and teaching environment. Together with teachers’, trainers’ and all other relevant stakeholders’ feedback this is used to inform further actions.

• Information on the outcomes of the review is widely and publicly available.

• Procedures on feedback and review are part of a strategic learning process in the organisation, support the development of high-quality provision, and improve opportunities for learners.

• Results/outcomes of the evaluation process are discussed with relevant stakeholders and appropriate action plans are put in place.

Countries adopting the EQAVET framework tend to use its indicative descriptors for the review phase the least, compared to the other phases (EQAVET, 2023[6]). Most of the systems analysed do, nevertheless, include some requirements for providers to use the results of the evaluation to implement improvement measures (Table 6.1). Other indicators related to the implementation of the review phase are the dissemination of the evaluation results, the implementation of review procedures and the involvement of stakeholders in the review phase. These quality criteria are described below.

Table 6.1. Quality criteria covered by the different QA systems

Criteria	EQAVET	CHE EduQua	SVN OQEA	NLD NRTO	AUT Ö-CERT (incl. QMS)	IRL QQI	FRA Qualiopi	PRT DGERT	ICE EQM	LUX Label de Qualité	ISO 21001/ 9001
Dissemination of evaluation results	x		x			x	x
Review procedures	x		x	x	x (QMS)	x	x		x
Develop an improvement plan	x	x	x	x	x (QMS)	x	x	x	x
Involve stakeholders	x		x	x		x			x	x

Notes: This table presents the QA systems by their names and Ireland, the Netherlands and Portugal by the entity that is responsible for QA, as the frameworks they developed do not have a name. Since Ö-CERT is an “umbrella label”, providers must prove that they have one of the 12 accepted quality management systems (QMS) to be awarded the Ö-CERT quality label. For this reason, Ö-CERT has been analysed jointly with four of the accepted QMS: Cert NÖ, EduQua, ISO 21001 and ISO 9001. Quality areas and criteria that include (QMS) in the Ö-CERT column are covered by accepted quality labels and not by Ö-Cert directly. Each “x” in the table indicates that the given QA system includes indicators in the corresponding criterion.

Source: Author’s elaboration.

As an intermediate step between the evaluation and the review phases, many of the systems reviewed for this report disseminate or require training providers to disseminate evaluation results. Making evaluation results public strengthens transparency of the quality of the services and training providers’ accountability. Trainers, trainees, employers, and accreditation bodies can gain a better understanding of the training provider’s efforts to ensure quality and of the quality of the services they currently provide. Publishing the results also holds training providers accountable by allowing stakeholders to compare the provider’s performance against established standards and benchmarks or against the performance of other providers. QQI, for example, requires the publication of quality assurance evaluation reports by training providers. Similarly, Qualiopi obliges training providers to disseminate their results in outcome indicators and to advertise the availability of this information in a specific dissemination website.

To ensure that the evaluation results are used to promote change, several of the QA systems reviewed require providers to implement review procedures. For example, Qualiopi requires training providers to address difficulties encountered by stakeholders during the training process as well as to review its assessments and complaints.

Most systems reviewed also require providers to use the evaluation of their services to draw up an improvement plan. This plan intends to improve the quality of the training services provided by systematically addressing the identified areas for improvement. In addition, if the improvement plan is, at least partly, based on the feedback of relevant actors, it shows stakeholders that training providers take their feedback into consideration while also holding training providers accountable in front of stakeholders. Ideally, this plan should be part of a continuous improvement system, which should be updated according to the latest evaluation results. In Ireland, for example, QQI expects all providers to use their annual self-evaluation outcomes to produce a quality improvement plan, which includes a schedule for the implementation of improvement actions and identifies responsibilities and follow-up activities. EduQua requires training providers to use the identified weaknesses and strengths of their training programmes to suggest further developments to their offer, while Qualiopi expects providers to demonstrate the implementation of a continuous improvement process.

Finally, in line with the EQAVET indicative descriptors, some systems involve several stakeholders beyond a one‑off survey to discuss the findings of the evaluation phase and potential improvements. EduQua, for example, encourages training providers’ management to be in constant communication with staff and other stakeholders so that everyone involved in the training process reflects on the evaluation results. In the OQEA model, stakeholders should be able to suggest improvement actions to the training provider. EQM in Iceland goes one step further, as it expects training providers to be constantly working on improving their services jointly with all stakeholders involved in the training process.

To verify compliance with these criteria and with the review phase more generally, QA frameworks rely on the documentation outlined in Table 6.2.

Table 6.2. Evidence used for the review phase quality criteria

Quality criteria	Evidence
Dissemination of evaluation results	All media and information tools Activity reports
Review procedures	Description of the review procedures
Develop an improvement plan	Identification and reflection on the causes of trainee abandonment or the reasons for dissatisfaction Action plans for improvement Implementation of specific actions
Involve stakeholders	Evaluation or feedback concept Example of a teaching or learning process evaluation with feedback to trainers Feedback instruments used

Source: Author’s elaboration.