Mid‑Term Evaluation of Spain’s Youth Guarantee Plus Plan 2021‑2027

Annex B. Evaluation matrix

Table A B.1. Evaluation matrix

ToC level/ evaluation criteria / evaluation question	Sub-questions	Indicators	Information sources and tools
The YG+P as a whole
Overall relevance of the programme
1. Do the objectives and design of the YG+P respond to Spain’s needs and priorities in relation to youth employment?		Macroeconomic analysis of key youth employment indicators, and profiling of NEETs in Spain. Comparison with the profiles of Youth Guarantee participants	Macroeconomic data and programme documentation Youth Guarantee Administrative Registry
YG+P expected outputs1
Line 1. Career guidance
Coverage 3. Does the career guidance system provide personalised support to young people, in particular to the most vulnerable groups? 4. [Line 4] Does the system ensure equality of access to that support?		Proportion of young people who received personalised career guidance services Proportion of vulnerable young people who received personalised career guidance services	Youth Guarantee Administrative Registry
Relevance 5. Is the support provided timely, of high quality and inclusive?	5.1. Is support provided in a timely manner?	Proportion of young people who received personalised career guidance services within four months of registering Proportion of vulnerable young people who received personalised career guidance services within four months of registering	Youth Guarantee Administrative Registry
	5.2. Is the support provided by qualified professionals?	Proportion of career guidance staff who have received specific training leading to a qualification in career guidance Young people’s perception of counsellors’ competence	Surveys of the regional PES and IBs Surveys of YG+P participants
	5.3. [Line 4] Is support provided by staff trained in equality in access to employment?	Proportion of career guidance staff who have received training in equality in access to employment	Surveys of the regional PES and IBs
	5.4. Are mechanisms in place to improve the quality and inclusiveness of career guidance?	Existence of mechanisms to improve the quality and inclusiveness of the career guidance system	Surveys of the regional PES and IBs
6. Is the support adapted to the personal characteristics and objectives of the young people and to the needs of the labour market?	6.1. Is the support adapted to the young people’s characteristics (including their level of education) and personal objectives?	Young people’s perception of the extent to which career guidance is adapted to their characteristics (including level of education) and personal objectives	Surveys of YG+P participants
	6.2. Is the support adapted to the needs of the labour market?	Collaboration with the business community on career guidance Adaptation of the career guidance to the needs of the labour market Young people’s perception of the extent to which career guidance is adapted to the needs of the labour market	Surveys of the regional PES and IBs Surveys of YG+P participants
Line of action 2. Training
Coverage 7. Are training programmes provided to young people, especially the most vulnerable groups? 8. [Line 4] Does the system ensure equality of access to that training?		Proportion of young people who received training Proportion of vulnerable young people who received training	Youth Guarantee Administrative Registry
Relevance 9. Is the training provided timely, of high quality and inclusive?	9.1. Are training programmes provided in a timely manner?	Proportion of young people who received training within four months of registering Proportion of vulnerable young people who received training within four months of registering	Youth Guarantee Administrative Registry
	9.2. Are the training programmes delivered by qualified professionals?	Proportion of training staff who have received specific training leading to a qualification in this domain Young people’s perception of training staff’s competence	Surveys of the regional PES and IBs Surveys of YG+P participants
	9.3. [Line 4] Are training programmes led by staff trained in equality in access to employment?	Proportion of training staff who have received training in equality in access to employment	Surveys of the regional PES and IBs
	9.4. Are there mechanisms in place to improve the quality and inclusiveness of training?	Existence of mechanisms to improve training quality and inclusiveness	Surveys of the regional PES and IBs
10. Does the training respond to the needs of the young people and align with labour market needs?	10.1. Are the training programmes adapted to the young people’s characteristics (including their level of education) and personal objectives?	Young people’s perception of the extent to which training is adapted to their characteristics (including their level of education) and personal objectives	Surveys of YG+P participants
	10.2. Do the training programmes align with labour market needs?	Collaboration with the business community on training Adaptation of training to labour market needs Young people’s perception of the extent to which training is adapted to labour-market needs	Surveys of the regional PES and IBs Surveys of YG+P participants
Lines 3 and 5. Employment and entrepreneurship opportunities
Coverage 11. Are employment and entrepreneurship opportunities offered to young people, and especially to the most vulnerable groups? 12. [Line 4] Does the system ensure equality of access to the opportunities generated?		Proportion of young people who received employment opportunities Proportion of vulnerable young people who received employment opportunities	Youth Guarantee Administrative Registry
13. Are partnerships and incentives in place to create and maintain employment opportunities for young people?	13.1. Are partnerships in place with the business community to expand the offer of first professional experiences for young people?	Collaboration with the business community to expand the offer of first professional experiences for young people	Surveys of the regional PES and IBs
	13.2. Are incentives and grants made available to the business community to support hiring and entrepreneurship?	Disbursement of incentives and grants for hiring and entrepreneurship	Surveys of the regional PES and IBs
	13.3. [Line 4] Are incentives and grants made available to the business community to ensure equality of access to employment and equal opportunities in regard to keeping a job?	Disbursement of incentives and grants to promote equality of access to employment and equal opportunities in regard to keeping a job	Surveys of the regional PES and IBs
Relevance 14. Are partnerships and incentives targeted to promote the hiring of, and entrepreneurship among, vulnerable young people? 15. Do the employment services respond to the needs of the young people?		Perception among PES staff of the extent to which the partnerships and incentives are targeted to benefit vulnerable young people Young people’s perception of the quality of the employment opportunities	Surveys of the regional PES and IBs Surveys of YG+P participants
Line 6. Improvement of the institutional framework
15. Have there been campaigns or events to disseminate information about, and raise awareness of, the YG+P?		Campaigns or events to disseminate information about, and raise awareness of, the YG+P	Surveys of the regional PES and IBs
16. Have internal and external co‑ordination systems been implemented to ensure the programme runs smoothly?	16.1. To what extent is the work of the different lines of action co‑ordinated (e.g. Line 1 with all the other lines; Line 3 with Line 5; Line 4 in a cross-cutting manner) to address common challenges?	Co‑ordination mechanisms are in place between professionals from the different lines of action. For example, good practices and lessons learned are shared to strengthen equality in access to employment	Surveys of the regional PES and IBs
	16.2. To what extent do the PES co‑ordinate with external actors to achieve programme objectives?	Co‑ordination mechanisms are in place with the business community, training centres, governmental organisations, etc.	Surveys of the regional PES and IBs
	16.3. Has a monitoring, evaluation and learning system been implemented for the YG+P?	Annual collection and presentation of indicator-framework monitoring results (at the output level) Commissioning of an independent mid-term evaluation of the YG+P Workshops for reflection and feedback	Surveys of the State Public Employment Service (SEPE)
Expected outcomes of the YG+P
Outcome 1. Young people’s employability is increased – Effectiveness
17. Does the personalised career guidance facilitate programme participants’ access to educational or training opportunities?		Econometric analysis	Youth Guarantee Administrative Registry
18. Does the training provided effectively improve participants’ skills and competencies?		Econometric analysis	Youth Guarantee Administrative Registry
Outcome 2. Decent and inclusive employment opportunities are generated for young people – Effectiveness
19. Do the employment and entrepreneurship services effectively generate decent medium-term employment and entrepreneurship opportunities for young people?		Econometric analysis	Youth Guarantee Administrative Registry

Note: 1. The OECD defines (OECD, 2002[1]) outputs (or first-level outcomes) as the products, capital goods and services that result from an intervention. They differ from outcomes in that they are within the control of the implementing team and attributable to it.

References

[1] OECD (2002), OECD Glossary of Key Terms in Evaluation and Results based Management Terminology, https://unsdg.un.org/sites/default/files/OECD-Glossary-of-Key-Terms-in-Evaluation-and-Results-based-Management-Terminology.pdf.