Improving Active Labour Market Policies in Spain

2. Evidence‑based design of ALMPs

Abstract

This chapter examines how evidence was used to inform the design of selected ALMPs in Spain. It begins by explaining why evidence‑based design is a key element of effective ALMPs, along with the benchmark used for the qualitative assessment. The chapter then presents the main findings of the assessment exercise, focussing on whether programmes drew on labour market data, lessons from past interventions, and stakeholder input during the design phase, and whether monitoring and evaluation frameworks were integrated from the outset. It then highlights two good practices and offers recommendations emerging from the assessment to strengthen the use of evidence in future ALMP design.

2.1. Understanding the criterion and its benchmark for qualitative assessment

A fundamental aspect of a well-designed programme is the incorporation of existing evidence. This approach ensures that policies are grounded in what has been proven to work, thereby mitigating the risk of inefficacy or unintended consequences. Effective programme design relies on a solid foundation of empirical data derived from systematic observation, measurement, experimentation, or evaluation, rather than theoretical predictions or anecdotal evidence. Evidence includes an analysis of historical trends, current labour market conditions, and the effectiveness of similar programmes in similar or other contexts. Programme design should involve a comprehensive review of past programmes, identifying what worked, what failed, and the reasons behind these outcomes. By harnessing evidence, particularly from previous robust monitoring and evaluation (M&E) exercises, policy designers can create more targeted and impactful ALMPs.

Additionally, benchmarking ALMPs with those in other OECD countries can offer fresh perspectives for policy design. This comparative analysis helps in identifying best practices and innovative approaches that have been successful elsewhere, potentially adapting them to the specific context.

Effective and efficient programme design also relies on formal and informal co‑operation between key stakeholders involved in the ALMP system (Lauringson and Lüske, 2021[6]). Such collaboration ensures a holistic approach, incorporating various perspectives and expertise. For example, involving social partners can yield valuable insights into the needs of jobseekers, employees, and employers, thereby fostering more valid, acceptable, and effective ALMP design. Denmark exemplifies how the input from social partners significantly influences policy design (OECD, 2021[7]). Moreover, co‑operation and co‑ordination between the stakeholders in ALMP systems have been crucial for quick and well-designed responses to the challenges posed by the COVID‑19 pandemic. In Austria, for instance, researchers were involved in re‑designing the ALMP package since the very beginning of the crisis through a standing research committee.

Lastly, designing policies in tandem with their monitoring and evaluation frameworks is crucial. This ensures that variables for data collection, data collection mechanisms, staff and skill requirements, and potential evaluation methodologies are considered in advance, making subsequent monitoring and evaluation activities more feasible. By establishing performance indicators and evaluation criteria in advance, an ongoing feedback loop can be maintained throughout the programme’s implementation, fostering continuous improvement and adaptation based on real-time data and outcomes (see Chapter 9 on monitoring and evaluation).

Box 2.1. Benchmark for excellence: ALMP design

A benchmark ALMP stands out for its evidence‑based design, informed by solid facts and findings. It is based on a comprehensive analysis of historical trends, current labour market conditions and challenges, empirical data and the lessons learned from other programmes. It learns from the results of previous and similar programmes, identifying what worked, what failed, and the reasons behind these outcomes. This approach ensures that programmes are grounded in what has been proven to work, thereby mitigating the risk of inefficacy or unintended consequences.

The design process involves collaboration with a range of stakeholders, including government bodies, employers, and employee representatives, and researchers. This ensures that the programme considers a diverse range of perspectives and needs. Additionally, this programme looks at successful ALMPs in other countries and uses these global insights for local use.

Furthermore, the best practice is designed alongside with its monitoring and evaluation (M&E) plan. This simultaneous development ensures that the programme establishes clear performance indicators and evaluation criteria from the beginning, which makes it easier to keep track of progress and effectiveness later on.

2.2. Summary of the qualitative assessment results

2.2.1. Use of evidence in programme design

The programmes assessed usually had three layers in their design. Initially, the broad content and objectives of the programmes were set at the national level. Subsequently, if regions were responsible for the management they could further refine and detail this framework. Finally, service providers contributed by suggesting specific actions and methodologies within the established programme guidelines.

A good practice observed in the design of the broad content at the national level was the use of lessons learned from previous technical assistance projects and expert consultations, such as the TSI project “Support to the implementation and monitoring of the measures included in the Youth Employment Shock Plan and in the Reincorpora-t Plan“, supported by the European Commission and the World Bank. These efforts helped identify effective approaches from previous interventions, shaping the development of programmes targeting vulnerable groups. Drawing on these insights ensured that the national-level content and objectives of the programmes incorporated elements that had been tested and validated in similar contexts.

At the regional level, most programmes built upon past experiences, particularly in areas such as counselling, training, and personalised itineraries, to refine and improve service delivery. Some programmes explicitly integrated evidence from past ALMPs to strengthen their design. In some cases, previous successful initiatives were adapted and expanded, incorporating elements such as more comprehensive support services, enhanced participant incentives, or stronger links with employers. In other instances, regions sought to replicate effective models that had been successfully implemented elsewhere, either in other regions or at the national level.

For certain programmes, particularly those targeting women victims of gender-based violence and human trafficking, the lack of previous large‑scale initiatives meant that programme design relied more heavily on the expertise of service providers. Given their specialised knowledge and hands-on experience, these providers played a key role in shaping the programmes, ensuring that interventions reflected the needs and realities of the target population.

Labour market data was also used to analyse trends and define target groups more precisely. Data from National and regional statistics institutes, employment observatories, and other data sources helped identify which groups faced the highest barriers to employment and where interventions were most needed. In some cases, data from past programme monitoring efforts – such as those conducted under the European Social Fund (ESF) – provided useful insights. However, the application of these monitoring results in designing new programmes was not systematic, and rigorous evaluations, such as counterfactual impact evaluations, remained rare. This lack of structured evidence limits the ability to identify and replicate the most effective programme elements, reducing the potential for continuous improvement in ALMP design.

2.2.2. Stakeholder collaboration in programme design

The extent of stakeholder involvement in programme design varied significantly across the assessed programmes. In most cases, PES led the design process with limited external consultation, primarily due to time constraints in programme development. Even collaboration within PES itself was not always present. For example, regional PES provide confidential support services for victims of gender-based violence, facilitated by specialised counsellors. While these dedicated services exist, collaboration between these specialised units and the PES teams responsible for designing and implementing the new programmes under the RRP was infrequent. Exceptions where this collaboration did happen proved highly beneficial, making the interventions more relevant and effective in addressing the specific needs of this group. The lack of structured collaboration with external stakeholders and across different PES departments and units, limits the ability of ALMPs to fully align with participant needs and labour market realities.

However, in some programmes, stronger stakeholder engagement was observed, particularly through social dialogue structures and advisory bodies. Some regions formally consulted trade unions and employer organisations before finalising programme designs, ensuring that the perspectives of both workers and businesses were reflected. In a few cases, social dialogue councils played a role in reviewing programme proposals, allowing social partners to influence decisions on target groups, service models, and support measures.

Dialogue and collaboration between PES and service providers were also instrumental in programme design. Consulting organisations working with specific vulnerable groups, such as NGOs and social services, was a particularly effective practice, helping to adapt activation strategies to the realities of jobseekers facing significant barriers. Additionally, in programmes targeting women victims of gender-based violence and human trafficking, law enforcement agencies and gender equality bodies were involved in the design phase, providing insights into the specific protection and integration challenges faced by women victims of violence.

2.2.3. Integration of monitoring and evaluation frameworks in programme design

All programmes incorporated some form of M&E framework at the design stage, primarily through the monitoring system established under the RRP (see Chapter 9). This ensured that basic tracking mechanisms were in place from the outset to monitor participant progress and programme implementation. However, the RRP framework primarily serves as a monitoring tool and lacks a structured evaluation component. While it facilitates the collection of quantitative data on participant flows and completion rates, it does not include mechanisms to assess whether observed outcomes – such as employment after programme completion – are directly attributable to participation rather than other external factors.

Beyond the RRP framework, some programmes integrated additional M&E design elements to improve tracking and accountability. In some cases, regions designed their own data collection tools, expanding the type of information gathered beyond the standard RRP indicators. Another good practice was the inclusion of operational guidelines for service providers, ensuring that data collection and reporting processes were structured and aligned with programme objectives. Some programme designs also incorporated structured follow-up mechanisms, requiring service providers to maintain participant records beyond programme completion, facilitating longer-term outcome tracking.

2.3. Good practices identified

The following two examples, presented in Box on good practices 1 and Box on good practices 2, stand out for aligning closely with the benchmark for evidence‑based ALMP design. The first example illustrates how the regional employment service in Extremadura systematically incorporated monitoring data, participant feedback, and service provider insights from earlier programme cycles to refine its support for vulnerable groups. The second highlights how the TIC and TURISMO programmes managed by SEPE in collaboration with FUNDAE embedded structured social dialogue and sectoral input into every stage of programme design. These cases offer concrete illustrations of how empirical evidence and stakeholder collaboration can lead to more responsive and well-targeted ALMPs.

Box on good practices 1. ALMP design for territorial projects for vulnerable groups in Extremadura

Empirical basis for programme design

The programme’s design was informed by the results of previous ALMPs conducted by the Extremaduran Public Employment Service (SEXPE) for job seekers with disabilities. Specifically, insights were gathered from programmes initiated in 2018 and 2021, each spanning 12 months and implemented through competitive grant procedures for non-profit organisations within the region. These programmes had counselling and professional guidance as its main component, but they also included upskilling actions.

The design process was informed by the detailed monitoring of these past programmes. The analysis focussed on the extent to which the set objectives were achieved, as described in comprehensive reports on the execution of actions provided by service providers. These final reports included an examination of the participants’ profiles, including age, gender, type of disability, and the targeted occupations. Moreover, the reports assessed the main activities of the programmes, pinpointing areas of success and failure. They helped identify potential areas for improvement, ensuring that each iteration of the programme was more finely tuned to meet its objectives.

Stakeholder collaboration

The design of the programme targeted at vulnerable groups in Extremadura was notably enhanced by the active involvement of various stakeholders, which included service providers implementing previous similar programmes, as well as the direct users of the programmes.

Service providers were key in providing insights into the barriers to employability faced by individuals with disabilities. Their feedback was crucial in identifying and refining areas of the programme needing enhancement. For example, feedback from the 2018 programme cycle indicated a significant gap in employer awareness and understanding of disability issues. This led to the introduction of supported employment actions in the 2021 programme, which offered personalised advice and support to help individuals with disabilities and special difficulties find, retain, and progress in mainstream employment in the open labour market. Additionally, based on feedback from these providers, significant enhancements such as the inclusion of actions to develop generic and transversal competencies, along with digital skills, were integrated into the new programme.

Participants also had a substantial influence on its design through their feedback. Surveys conducted among participants, offered direct insights into their satisfaction levels and the relevance of the skills they acquired.

Monitoring and Evaluation Design

The monitoring and evaluation framework in Extremadura was designed from the outset and extends beyond the standard M&E framework common to all programmes for vulnerable groups developed under the Recovery and Resilience Mechanism (RRM). Key to this approach was the enhancement of an existing SEXPE computer application, initially used for basic data collection, which now captures a broader range of both quantitative and qualitative data. This enhanced application is intended to become a regular tool for SEXPE, with future developments planned to allow for more in-depth analysis of the qualitative and textual data gathered.

Additionally, the M&E design is supported by the “Manual for Action Development,” provided by the SEXPE to all service providers. This manual offers instructions on all actions to provide within the programme and how to monitor them effectively. It serves as an essential tool for ensuring consistent and thorough tracking of the programme’s progress and effectiveness. This combination of the design of a data collection tool and clear operational guidance establishes a robust framework for M&E.

Source: Authors from information collected through questionnaires and consultations.

Box on good practices 2. Stakeholder collaboration in the design of the TIC and TURISMO programmes throughout the territory (SEPE FUNDAE)

The design of the TIC and TURISMO programmes, managed by SEPE in collaboration with the State Foundation for Training in Employment (FUNDAE), exemplifies a good practice, showcasing an evidence‑based approach underpinned by robust stakeholder collaboration. This practice is deeply embedded in FUNDAE’s structure, where the integration of a wide array of perspectives ensures that the programmes are well-aligned with current labour market needs and trends

Governing board composition – FUNDAE

The governing board of FUNDAE plays a pivotal role in the approval and strategic oversight of training programmes. It comprises representatives from the General State Administration and regions including Andalucía, Aragón, Asturias, Baleares, Canarias, Cantabria, Castilla y León, Castilla-La Mancha, Cataluña, Extremadura, Galicia, La Rioja, País Vasco, Madrid, Murcia, Navarra, and Valencia. The board also features the most representative trade unions and employer organisations. These include the Confederación Sindical de Comisiones Obreras (CCOO), Unión General de Trabajadores (UGT), Confederación Intersindical Galega (CIG), Confederación Española de Organizaciones Empresariales (CEOE), and Confederación Española de la Pequeña y Mediana Empresa (CEPYME). This diverse composition ensures that a broad range of regional and economic interests are considered in decision making.

Role of sectoral commissions (“Comisiones Paritarias Sectoriales”)

At the heart of FUNDAE’s collaborative framework are the sectoral commissions, which are crucial in the design, programming, monitoring, and evaluation of training programmes. These commissions, formed through agreements in state‑level collective bargaining, comprise representatives from trade unions and employer organisations. They identify training needs directly from the workplace, ensuring that the training provided is responsive to the evolving demands of the economic sectors.

Over three decades, these commissions have significantly contributed to the design and dissemination of training programmes across various sectors. Currently, there are 91 sectoral joint commissions and one sector-specific paritary structure for the digital economy, highlighting the extensive coverage and specialised attention to detailed sectoral needs. Their main function is to establish sectoral training priorities, which are then incorporated into a sectoral reference plan detailing the training specialties available for each sector through state‑level on the job training programme calls.

These commissions also play a mediating role in resolving disputes between companies and legal worker representatives over scheduled training, ensuring smooth implementation and adaptation to on-ground realities. Furthermore, they actively participate in trend studies and training evaluations, offering insights into sectoral evolutions and potential skill gaps.

Good practice in ALMP design

The good practice in the design of ALMPs like TIC and TURISMO through the co‑operation with FUNDAE is evident in its foundation on empirical analysis, historical trends, and the real-time challenges of the labour market. By drawing on the lessons learned from other programmes and incorporating feedback from a wide range of stakeholders, these programmes not only adhere to the benchmarks for good practice but also set a standard for incorporating comprehensive stakeholder insights, significantly reducing the risk of inefficacy from the design stage.

Source: Authors from information collected through questionnaires and consultations. https://blog.fundae.es/aprendizaje‑permanente/2024/01/29/comisiones-paritarias-sectoriales-sistema-formacion-empleo/.

2.4. Policy directions: Avenues for future ALMP design

2.4.1. Strengthen the use of empirical evidence in programme design

While many programmes build on past experiences, the systematic use of empirical evidence remains limited. Building on the foundations laid by the Spanish Strategy for Active Support to Employment (Estrategia Española de Apoyo Activo para el Empleo, EEAAE (2021‑2024)), which promotes the use of evidence and data in ALMP planning, future ALMPs should:

• Ensure that new programme designs systematically incorporate learnings from previous ALMPs, particularly by leveraging existing monitoring data and evaluations to refine interventions.

• Enhance the use of labour market data analysis to inform ALMP design, including defining target groups, service offerings, and programme scope based on employment trends, occupational demand, and local economic conditions.

• SEPE can play a key role in facilitating cross-regional peer learning, enabling regions to exchange on their practices and lessons learned and scale up effective ALMP models, ensuring that successful approaches are adapted and implemented more widely. Existing mechanisms such as the Network of Public Centres for Counselling, Entrepreneurship Support and Innovation for Employment (Red de Centros Públicos de Orientación, Emprendimiento, Acompañamiento e Innovación para el Empleo, Red COE) can support this exchange by systematising the identification and dissemination of good practices across the National Employment System.

2.4.2. Enhance stakeholder collaboration in programme design

Stakeholder engagement in programme design was often limited, reducing the alignment of ALMPs with labour market realities. Future programmes should:

• Strengthen collaboration across departments and units within PES and ensure the involvement of specialised teams and employment counsellors in programme design, particularly those working directly with specific jobseeker groups (e.g. women victims of gender-based violence, long-term unemployed, etc.).

• Ensure that consultations with employer organisations and trade unions systematically inform programme design at all levels, to better align training and employment services with real workforce demands and worker needs. At the state level, structured consultations already take place through the General Council of the National Employment System (Consejo General del Sistema Nacional de Empleo), a tripartite consultative body established under Article 10 of Law 3/2023. However, for programmes managed at the regional level or for design elements under the responsibility of regional PES, such consultations are not always systematic. These consultations should be initiated during the early stages of programme design and co‑ordinated by the PES units responsible for programme planning and design. Leveraging existing social dialogue structures or establishing dedicated advisory meetings can ensure that stakeholder input informs key design elements such as target groups, programme content (e.g. training), and delivery approaches.

• Ensure that service providers, NGOs and social services are engaged from the outset, particularly in programmes targeting vulnerable groups, to design interventions that are tailored to their specific needs and circumstances. PES should initiate this engagement through structured consultation formats- such as joint planning workshops or targeted focus groups- co‑ordinated by programme designers.

2.4.3. Incorporate M&E framework design from the outset

While all programmes included some form of monitoring, evaluation was rarely integrated at the design stage. Building on the integrated evaluation model outlined in the EEAAE (2021‑2024) and its implementation through Annual Employment Policy Plans, further efforts are needed to ensure that evaluation planning is systematically embedded from the outset and fully operationalised across the National Employment System. Future ALMPs should:

• Establish clear guidelines for data collection and reporting, ensuring consistency across geographical areas and service providers. These guidelines should define the types of data to be collected and ideally incorporate both quantitative and qualitative data on participant outcomes during and beyond programme completion (for at least 12 months), as well as measures to capture broader dimensions such as well-being and social integration.

• Move beyond basic monitoring by embedding evaluation components from the outset, defining a clear evaluation plan to assess programme impact, effectiveness, and efficiency. This should include specifying evaluation objectives and indicators during the design phase, selecting appropriate methodologies (e.g. counterfactual impact evaluations), identifying responsible actors (internal or external), and allocating sufficient resources to carry out the evaluation.